-- Logs begin at Tue 2022-06-28 11:18:42 UTC, end at Tue 2022-06-28 12:28:22 UTC. -- Jun 28 11:50:12.265674 dv-cinder-847954-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 28 11:50:12.274701 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 28 11:50:12 2022] *** Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: nodename: dv-cinder-847954-1-iscsi Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: machine: x86_64 Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: clock source: unix Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: pcre jit disabled Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: detected number of CPU cores: 4 Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: current working directory: / Jun 28 11:50:12.276734 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: detected binary path: /usr/bin/uwsgi-core Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: your processes number limit is 47680 Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: your memory page size is 4096 bytes Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: detected max file descriptor number: 2048 Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: lock engine: pthread robust mutexes Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: thunder lock: enabled Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 28 11:50:12.277507 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 28 11:50:12.320779 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: Python main interpreter initialized at 0x562e5ff2c9c0 Jun 28 11:50:12.320779 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: python threads support enabled Jun 28 11:50:12.320779 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: your server socket listen backlog is limited to 100 connections Jun 28 11:50:12.320779 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: your mercy for graceful operations on workers is 90 seconds Jun 28 11:50:12.321182 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: mapped 403077 bytes (393 KB) for 2 cores Jun 28 11:50:12.321182 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: *** Operational MODE: preforking *** Jun 28 11:50:12.321493 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: *** uWSGI is running in multiple interpreter mode *** Jun 28 11:50:12.321493 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: spawned uWSGI master process (pid: 80085) Jun 28 11:50:12.321671 dv-cinder-847954-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 28 11:50:12.325388 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: spawned uWSGI worker 1 (pid: 80099, cores: 1) Jun 28 11:50:12.325388 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: spawned uWSGI worker 2 (pid: 80100, cores: 1) Jun 28 11:50:12.325388 dv-cinder-847954-1-iscsi devstack@keystone.service[80085]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 28 11:50:17.772010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 28 11:50:17.772010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: from cryptography.utils import int_from_bytes Jun 28 11:50:17.772734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 28 11:50:17.772734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: from cryptography.utils import int_from_bytes Jun 28 11:50:17.773282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 28 11:50:17.773282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: from cryptography.utils import int_from_bytes Jun 28 11:50:17.773404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 28 11:50:17.773404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: from cryptography.utils import int_from_bytes Jun 28 11:50:17.821435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 28 11:50:17.821637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 28 11:50:17.821721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 28 11:50:17.821801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 28 11:50:17.821801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] command line args: [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 28 11:50:17.821918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] command line args: [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 28 11:50:17.822064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 28 11:50:17.822149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 28 11:50:17.822215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ================================================================================ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 28 11:50:17.822274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ================================================================================ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 28 11:50:17.822437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] admin_token = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.822537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] admin_token = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.822662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_dir = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.822747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_dir = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.822857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.822941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] config_source = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] config_source = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] debug = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] debug = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.823949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_config_append = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_config_append = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.825945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_dir = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_dir = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.826970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.827983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.828826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] publish_errors = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] publish_errors = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.829969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.830950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] standard_threads = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] standard_threads = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_journal = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_journal = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.831927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_json = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_json = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_stderr = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_stderr = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] use_syslog = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] use_syslog = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 28 11:50:17.832631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.832714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.832827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.832935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.833872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.834813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.835923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.836945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.837870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.838961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.839948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.840888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.841789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.842966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.843956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.844948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.845124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.845286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.845457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.845634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.845847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.846858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.847960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.848995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.849974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.850949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.851973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.852896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.853915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.854952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.855954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.856762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.857276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.external = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.857429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.external = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.857702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.857868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.857982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.password = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] auth.token = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.password = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.858956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] auth.token = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.859890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.860939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.861974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.862892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.863900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.864855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.865967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.866925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.867920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.868862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.869952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.870963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.871993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.872961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.873979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.874985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.875870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.876879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.877922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.878981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.879884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.880942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.881854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.882985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.883877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.884898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.885898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.886887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.887880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.888944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.889974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.890898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.891902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.892877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.893913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.894937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.895960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.896832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.897920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.898949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.899952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.900886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901761 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.901978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.902930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.903971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.904898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.905900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.906971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80100) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 28 11:50:17.907415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 28 11:50:17.907415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80099) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 28 11:50:17.937794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.938019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.938219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.938313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.938451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.938664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.946899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.947109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.974901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.975151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.975826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:17.976054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.122267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.125430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.171226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.171456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.171711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.171929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.236964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.237061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.237136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.237293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.261908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.262981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.263050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.263124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.263199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.263403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.274347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.274722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.287948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80100) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.288930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80099) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 28 11:50:18.305806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.306022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 28 11:50:18.306120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.306196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 28 11:50:18.306291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.306492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 28 11:50:18.306615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.306693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 28 11:50:18.306769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.306957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.307659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.318571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.318846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.319858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.327744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.331369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.331644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.331918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.332188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.332464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.332797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.333088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.333359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.333623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.333893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.334138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.335025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.338348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.338626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.338900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.339168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.339426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.339677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.339932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.340191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.340449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.340708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.340996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.361151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.362763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.363034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.368318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.368704 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.369004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.376521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.376741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.378312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.379967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.380236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.381270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.381554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.381832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.382106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.382375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.382639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.382902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.383161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.385708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.386107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.386391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.394042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.394325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.398849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.399134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.399410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.399686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.399956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.400227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.400494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.400757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.402164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.402440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.407181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.410851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.411245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.416265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.419678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.420009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.420109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.423626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.423920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.424197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.424472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.424581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.424748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.425054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.428288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.428691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.433764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.437575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.441248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.441542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.441824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.442101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.442388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.442677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.444533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.444823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.445107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.449071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.453539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.455245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.455526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.458829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.459124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.459420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.459714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.460021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.462067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.462347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.462610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.462869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.471085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.472797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.473079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.473414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.473695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.473969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.474244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.475908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.476197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.476339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.476502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.476643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.476841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.477148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.477420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.491755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.492029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.492314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.492591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.494312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.494603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.494896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.495021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.498407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.501958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.503672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.505323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.505579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.505887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.512994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.513257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.514994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.516347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.518265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.518549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.518938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.519212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.519687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.521417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.523071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.523354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.523642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.528322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.528605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.528887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.531055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.532784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.536066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.536354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.536753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.537059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.537407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80100) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.537683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.537961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.538234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.538502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.538870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.539158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.539408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.539671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.539946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.540211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.540464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.540733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.541001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80100) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.546318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.546602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.546885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.554782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80099) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 28 11:50:18.555062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.555339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.555587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.555866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.556237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.556524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.556793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.557086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.557377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.557646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.557889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.558147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.558401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80099) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 28 11:50:18.579083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562e5ff2c9c0 pid: 80100 (default app) Jun 28 11:50:18.582549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65b21-969a-4494-a792-a63db73c6c10 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:18.582839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65b21-969a-4494-a792-a63db73c6c10 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:18.583112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65b21-969a-4494-a792-a63db73c6c10 None None] PATH_INFO: `/v3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:18.587905 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Tue Jun 28 11:50:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:18.596598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562e5ff2c9c0 pid: 80099 (default app) Jun 28 11:50:26.488470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:26.489134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:26.489134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:26.490430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 2/2] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:26.500665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:26.501144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:26.501243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:26.569937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: INFO dbcounter [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] Registered counter for database keystone Jun 28 11:50:26.594641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG oslo_db.sqlalchemy.engines [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80099) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 28 11:50:26.606409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writer thread running {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 28 11:50:26.791541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG passlib.handlers.bcrypt [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80099) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 28 11:50:26.791964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG passlib.handlers.bcrypt [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80099) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 28 11:50:26.846891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:26.848501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:26.976010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1/3] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 28 11:50:27.161095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:27.242893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: INFO dbcounter [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] Registered counter for database keystone Jun 28 11:50:27.264867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG oslo_db.sqlalchemy.engines [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80100) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 28 11:50:27.273607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writer thread running {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 28 11:50:27.400252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ee83-78ef-484a-94cb-7a1477cff3ec None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:27.406006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:27.406776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:27.407059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:27.407340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:27.416114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:27.467496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 28 11:50:27.467834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431fba-041e-4feb-a643-c980f886832a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:27.479664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 3/4] 10.222.0.55 () {58 vars in 1324 bytes} [Tue Jun 28 11:50:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 11:50:27.489097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:27.505059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b7ead-2b7d-4b25-9607-3275b08d1dfd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:27.510421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:27.511574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:27.511857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:27.512137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] PATH_INFO: `/v3/projects/9c70d24580e841bb8af0388fc0999482` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:27.516444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] RBAC: Authorizing `identity:get_project(project_id=9c70d24580e841bb8af0388fc0999482)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:27.567520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 28 11:50:27.567861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377a95-f826-4c65-8376-c6aee51f8eb0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:27.572508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 2/5] 10.222.0.55 () {56 vars in 1207 bytes} [Tue Jun 28 11:50:27 2022] GET /identity/v3/projects/9c70d24580e841bb8af0388fc0999482 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:29.572445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47fa7-2bb1-4462-bd0f-cd7d9ef8d838 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:29.573377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47fa7-2bb1-4462-bd0f-cd7d9ef8d838 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:29.573377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da47fa7-2bb1-4462-bd0f-cd7d9ef8d838 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:29.580522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 4/6] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:29.587269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:29.587706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:29.587868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:29.610277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:29.611815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:29.621108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 3/7] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 28 11:50:29.644332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da47fa7-2bb1-4462-bd0f-cd7d9ef8d838 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:29.659518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da47fa7-2bb1-4462-bd0f-cd7d9ef8d838 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:29.664886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:29.667290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:29.667290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:29.667290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:29.675326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:29.677135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad227dc-4193-4670-af9e-18b376dc4fb2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:29.688087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 5/8] 10.222.0.55 () {58 vars in 1324 bytes} [Tue Jun 28 11:50:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 11:50:29.694262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:29.704594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7e8d3-2b9a-40f5-9786-efcfbc4db1d8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:29.711073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:29.711957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:29.712238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:29.712511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:29.714068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:29.715951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079587d-71dc-475f-8add-2432b67f681e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:29.717587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 4/9] 10.222.0.55 () {56 vars in 1198 bytes} [Tue Jun 28 11:50:29 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:31.846988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb1d8-f693-41b6-9416-06ad32a8bbcd None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:31.847569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb1d8-f693-41b6-9416-06ad32a8bbcd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:31.847569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb1d8-f693-41b6-9416-06ad32a8bbcd None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:31.848946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 6/10] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:31.854380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:31.854806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:31.854898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:31.865670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033ee21-f08d-4bed-8576-7ea0ebad3e94 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:31.865958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033ee21-f08d-4bed-8576-7ea0ebad3e94 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:31.866231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033ee21-f08d-4bed-8576-7ea0ebad3e94 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:31.867556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 7/11] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:31.872479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:31.872894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:31.873207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:31.873675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:31.875012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:31.912760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG passlib.handlers.bcrypt [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80100) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 28 11:50:31.913234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG passlib.handlers.bcrypt [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80100) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 28 11:50:31.958047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:31.959654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:31.959805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 5/12] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:31.985603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.010976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.065306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 8/13] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:50:32.087130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.090855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a29ba3-5c3a-4c3e-b593-4b83b19eac30 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.128193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.129033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.129311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.129577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.141815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:32.153031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:32.154117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:32.154969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 6/14] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:32.160712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.191193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff1dfb-0532-4316-845b-8887e1a86072 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.193596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.229538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.230453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.230740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.231014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.243946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.263601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941772b1-d4d2-44c4-8601-438b4a785f1e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.297488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.297760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.298156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.298437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.298706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.299318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.299485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:32.299762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.301186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.312850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 7/15] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:32.318623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.405525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 9/16] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 28 11:50:32.412636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.415497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.428010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da13f13-304b-44fe-a931-80bec267540e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.463311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.464696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.465079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.465390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.466768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.469099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.470758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 8/17] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:32.476628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.497534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.514405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b5ae9-e026-4557-a1c1-f5b94981aac0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.554326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.555143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.555420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.555690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.578473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.580202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.597792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75cfda-7dc5-4b23-a478-ae3db27d52ed None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:50:32.602613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:32.603559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:32.604385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:50:32.604385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:50:32.604385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:50:32.604385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:32.604385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 10/18] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:50:32.610485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.641911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.642719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.642993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.643258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.644452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.645956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.647582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 9/19] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:32.653421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:32.682468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.735959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09766d5-e911-43eb-9d31-9ecc61523b2e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.752421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:32.772556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda303a-6093-4895-83e6-2af65177bef9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:32.785367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.786283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.786565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.786831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.787691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:32.787996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.789491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe4889-3644-4904-b292-e39eff22c76d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.811130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:32.811797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:32.811997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:32.812090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:32.812378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:32.813254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.domain.id=default, user.id=02f93a0d97354be4b96d139a23cac2ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:32.813573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:32.815160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953bb76-be93-42fd-9887-47abc587d263 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:32.823658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1322 bytes} [Tue Jun 28 11:50:32 2022] GET /identity/v3/role_assignments?user.id=02f93a0d97354be4b96d139a23cac2ce&scope.domain.id=default&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 233 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:35.244623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5275-2505-470e-9066-893d94178cc0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.245144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5275-2505-470e-9066-893d94178cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.245144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a5275-2505-470e-9066-893d94178cc0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.246785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:35.252987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0a0e-f474-44a9-8ce2-7e946e28ac40 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.254029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0a0e-f474-44a9-8ce2-7e946e28ac40 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.254496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0a0e-f474-44a9-8ce2-7e946e28ac40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.282038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f43b0a0e-f474-44a9-8ce2-7e946e28ac40 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:35.283614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f43b0a0e-f474-44a9-8ce2-7e946e28ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:35.367805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 11/23] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:35.406483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a5275-2505-470e-9066-893d94178cc0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:35.458389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5872-05da-490c-babd-6e3043c786bf None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.459036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5872-05da-490c-babd-6e3043c786bf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.459512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5872-05da-490c-babd-6e3043c786bf None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.461137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 12/24] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:35.466354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.466873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.467356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.501988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:35.503481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:35.590333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a5275-2505-470e-9066-893d94178cc0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:35.649269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:35.668559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:35.669415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.669709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.669990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.688786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:35.694305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 13/25] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:50:35.698508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-20726fe1-31c6-4437-9688-08e2eb1b86fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:50:35.719040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:35.719944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:35.720745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:50:35.720745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:50:35.720745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:50:35.720745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:35.722732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 13/26] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:35.728862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:35.737313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:35.746325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b89a8-ee8d-4b6b-ba30-36f348a9003b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.747312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b89a8-ee8d-4b6b-ba30-36f348a9003b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.747773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b89a8-ee8d-4b6b-ba30-36f348a9003b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.750046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 14/27] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:35.751384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b89a8-ee8d-4b6b-ba30-36f348a9003b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:35.924691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b89a8-ee8d-4b6b-ba30-36f348a9003b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:35.952427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125d21f-dca6-4b83-962b-089e11a90770 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:35.980168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:35.981442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:35.982285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:35.982765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:35.997411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-16caa499-73bc-4bfb-8601-10221dbec695 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:36.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:36.012198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:36.014644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:36.020494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 15/28] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:36.022126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e05c-96f2-49f5-a374-755faf66188e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.022645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e05c-96f2-49f5-a374-755faf66188e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.023115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56e05c-96f2-49f5-a374-755faf66188e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.024419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.025125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.025485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.025773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.026620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:36.026927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.034642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.052082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aa56e05c-96f2-49f5-a374-755faf66188e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:36.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa56e05c-96f2-49f5-a374-755faf66188e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:36.058245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 14/29] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:36.058914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.081661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.140700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 16/30] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:36.145106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c690f2-301a-45ad-91cb-c83ccba11f2c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.145106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c690f2-301a-45ad-91cb-c83ccba11f2c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.145106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c690f2-301a-45ad-91cb-c83ccba11f2c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.145106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 17/31] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:36.145106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c690f2-301a-45ad-91cb-c83ccba11f2c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.153575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.205555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97261c71-d68e-4038-b2d4-80a3351086e7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.250414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.251551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.252286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.252727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.253700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:36.254627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.256995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3eb682-9dcc-472d-b874-08f045e030ec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.268460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 15/32] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:36.273001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c690f2-301a-45ad-91cb-c83ccba11f2c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.273487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.273487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.273487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.290869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:36.292338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:36.318230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.319615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.320112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.321043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.322498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.324948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.371939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 16/33] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:36.372854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.437819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.465964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 18/34] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/projects => generated 301 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:36.466990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.490584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.502821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f2116-3727-45e1-bc1d-14cca1ef75cd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.547503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.548565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.549086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.549577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.550602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.552230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.602846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3b58c-4968-48c4-994a-06cc5a62c9dc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.651343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.652247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.652553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.652834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.655390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.658075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a43b7-874f-45ee-b888-76488a1a3ec9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.660016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 19/35] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:36 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:36.665726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509adf-41b7-4561-a7d2-dc71d2a4d247 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.666008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509adf-41b7-4561-a7d2-dc71d2a4d247 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.666297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509adf-41b7-4561-a7d2-dc71d2a4d247 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.667865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 20/36] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:36.668610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509adf-41b7-4561-a7d2-dc71d2a4d247 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.699685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 17/37] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/roles => generated 219 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:36.700691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.771969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.803831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509adf-41b7-4561-a7d2-dc71d2a4d247 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.840050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcac046-6b30-452d-8cf6-443a77d04d25 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:36.853997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.854876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.855574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.855870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.857315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.858356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:36.858908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17c11-2a8c-4b95-b6e7-4c9dc9f7d7a4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.876627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 21/38] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:36.878020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9883-ae25-4566-b4d0-88e8f17dead7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.878310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9883-ae25-4566-b4d0-88e8f17dead7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.878590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9883-ae25-4566-b4d0-88e8f17dead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.885780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:36.887071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.887368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.887644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.889202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:36.891018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ac429-54ba-4468-9dc3-40b85f44d3c2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:36.893066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 18/39] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:36.894541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ceec6-8f7f-4114-a8f2-43909c00c771 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:36.894899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ceec6-8f7f-4114-a8f2-43909c00c771 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:36.895316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ceec6-8f7f-4114-a8f2-43909c00c771 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:36.901064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 19/40] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:36.902597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ed5c9883-ae25-4566-b4d0-88e8f17dead7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:36.903814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ceec6-8f7f-4114-a8f2-43909c00c771 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:36.905849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ed5c9883-ae25-4566-b4d0-88e8f17dead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:37.001707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 22/41] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:37.003425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:37.003938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:37.004413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:37.034067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:37.035315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:37.050104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ceec6-8f7f-4114-a8f2-43909c00c771 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:37.090228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:37.091051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:37.091339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:37.091616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] PATH_INFO: `/v3/domains/default/users/02f93a0d97354be4b96d139a23cac2ce/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:37.093869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=02f93a0d97354be4b96d139a23cac2ce, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:37.095345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:37.105872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:37.110581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 20/42] 10.222.0.55 () {58 vars in 1361 bytes} [Tue Jun 28 11:50:36 2022] PUT /identity/v3/domains/default/users/02f93a0d97354be4b96d139a23cac2ce/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:50:37.111388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:37.128739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 23/43] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:37.149006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:37.270002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f332a05-cf44-43a9-8ee3-ad4b64115c19 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:37.278686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:37.312392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:37.313955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:37.314251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:37.314533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:37.315783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:37.317749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:37.320249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 24/44] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:37.325741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:37.342891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8c6a-d169-411d-963f-0ed04ec87e8c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:37.386502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:37.387322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:37.387609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:37.387877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:37.388727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:37.390471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c4a4-c95c-420a-962a-f6b13b97ddea None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:37.396553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:37.416948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 21/45] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Jun 28 11:50:37 2022] POST /identity/v3/roles => generated 223 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:37.447940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae258f09-2bc6-4815-8001-0b4d48788465 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:37.487831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:37.488515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:37.488805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:37.489108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:37.489842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:37.491362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4073ef0-a42e-4236-95e4-0d157381860e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:37.520808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 25/46] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:50:37 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:50:39.245872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7d222-90da-48e3-b625-39c80a93763e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.246394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7d222-90da-48e3-b625-39c80a93763e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.246394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7d222-90da-48e3-b625-39c80a93763e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.247730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 22/47] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:39.253426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.253850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.254022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.271995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:39.273263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:39.342472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 26/48] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:39.366920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7d222-90da-48e3-b625-39c80a93763e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:39.473522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7d222-90da-48e3-b625-39c80a93763e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:39.475859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:39.511772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:39.512696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.518736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.519028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.541518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d1463c46-0921-4382-9691-be5f75e0f24a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:39.549925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:39.550787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:39.551543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 23/49] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:39.557400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:39.567897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71d09-48f4-4baf-8bc1-b9431069a5c6 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.568173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71d09-48f4-4baf-8bc1-b9431069a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.568452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71d09-48f4-4baf-8bc1-b9431069a5c6 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.569881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 24/50] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:39.574961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.575257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.575534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.593584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:39.595113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:39.612065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:39.663083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 25/51] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:39.665283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd13d-5d59-405c-b123-35fda8dbdcae None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:39.686707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:39.699479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:39.700158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.700441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.700723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.701512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:39.701824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:39.703325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:39.715081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 27/52] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:39.720673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:39.782685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:39.790494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a834e-0089-418e-9d65-fe9eaa5c6e53 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:39.824570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c7e8-c9a5-4f02-800d-b471590d1a5b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:39.824732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:39.825494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.825783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.826064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.827275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:39.828860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:39.830667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 26/53] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:39.849675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:39.953699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:39.972561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046fc08-7ee1-40e1-a67a-39f80584e1d6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:39.978904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:39.979816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:39.980098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:39.980375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:39.981928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:39.983393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:39.985465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 28/54] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:39 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:39.990683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:40.011493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:40.012399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:40.012680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:40.012998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:40.013723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:40.015176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:40.044133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 27/55] 10.222.0.55 () {60 vars in 1140 bytes} [Tue Jun 28 11:50:39 2022] POST /identity/v3/projects => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:50:40.097287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec9b8-3bd4-4685-9367-e92c96e089a6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:40.135041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:40.135874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:40.136158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:40.136438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:40.137724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:40.139209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472492f6-9850-4de6-9989-58ce049e5b96 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:40.141600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 29/56] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:40.147450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:40.166845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:40.267804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da124781-ace7-44fb-b501-a3ba809efa46 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:40.302122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:40.302709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:40.302990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:40.303262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:40.304094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.domain.id=default, user.id=02f93a0d97354be4b96d139a23cac2ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:40.304424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:40.305923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5388-e4f7-4f10-8c6e-10a59ddce1de None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:40.311693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 28/57] 10.222.0.55 () {56 vars in 1322 bytes} [Tue Jun 28 11:50:40 2022] GET /identity/v3/role_assignments?user.id=02f93a0d97354be4b96d139a23cac2ce&scope.domain.id=default&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:42.072174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0eda-420f-4be2-b945-abd3e036c93d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:42.072986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0eda-420f-4be2-b945-abd3e036c93d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:42.072986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0eda-420f-4be2-b945-abd3e036c93d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:42.075256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 30/58] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:42.083911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:42.084316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:42.084526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:42.104065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:42.105319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:42.179821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 29/59] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:42.204368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0eda-420f-4be2-b945-abd3e036c93d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:42.258351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:42.325436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0eda-420f-4be2-b945-abd3e036c93d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:42.361743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:42.362528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:42.362804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:42.363073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:42.364382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:42.365867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88498-787e-4975-a265-45e8501f9234 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:42.367482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 31/60] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:42.375740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:42.386276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:42.490384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0d2d2-6fcf-4c1a-bc74-17cd7540b7de None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:42.526741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:42.527414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:42.527698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:42.527976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:42.528682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:42.530185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352d08-c063-4824-b0d0-7cfdb6b6fe89 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:42.556709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 30/61] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:50:42 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:45.830204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3fc7-00f2-4837-8f7f-05c9f14bd1fa None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.830711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3fc7-00f2-4837-8f7f-05c9f14bd1fa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.830782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3fc7-00f2-4837-8f7f-05c9f14bd1fa None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:45.832552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 32/62] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:45.854719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29be9-43c7-4122-8308-82672e948ddb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.855213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29be9-43c7-4122-8308-82672e948ddb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.855314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab29be9-43c7-4122-8308-82672e948ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:45.858475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3182-5ab0-4dc9-8ea8-9af834dab683 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.861271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3182-5ab0-4dc9-8ea8-9af834dab683 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.861764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a3182-5ab0-4dc9-8ea8-9af834dab683 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:45.863406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 33/63] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:45.868770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.870601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.871086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:45.886970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-eab29be9-43c7-4122-8308-82672e948ddb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:45.893524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eab29be9-43c7-4122-8308-82672e948ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:45.910099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:45.911644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:45.968621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:45.978833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 31/64] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:45.980440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60c7-d4c7-49c4-bfbe-2cae620213ac None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.980876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60c7-d4c7-49c4-bfbe-2cae620213ac None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.981291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a60c7-d4c7-49c4-bfbe-2cae620213ac None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:45.983056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 32/65] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:45.989256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:45.989668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 34/66] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:45.989983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:45.990466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.008277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:46.009977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:46.010503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.065184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.093125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 33/67] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:50:46.098260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.160789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58baa88c-b6af-4963-adfc-b717ec645ea0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.204540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.205844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.206347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.206811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.218207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:46.219402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e134f3-a2be-411b-aec2-6214d92ff635 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:46.227392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:46.228269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:46.229060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:46.232134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 35/68] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:50:46.232731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.252797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.265414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.266288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.266610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.266886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.277060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:46.285360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:46.286201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:46.286958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 34/69] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:46.287582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.340844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3746a-c7db-4b18-ba5e-493fd13caa0a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.343155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.375724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.376536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.376817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.377117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.387617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:46.387870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f405fa5-c2f4-4aef-b391-f57bb34a236d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-19197870-cf65-447e-a55a-35fbb4fb976f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:46.396154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:46.397119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:46.397886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:46.401204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 36/70] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:46.402373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9f4d-5249-4dba-90c5-9020020df23b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.402662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9f4d-5249-4dba-90c5-9020020df23b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.402944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9f4d-5249-4dba-90c5-9020020df23b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.404489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 37/71] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:46.405576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fa1c-49fb-43d0-a634-b15fda49ab56 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.405964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fa1c-49fb-43d0-a634-b15fda49ab56 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.406278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5fa1c-49fb-43d0-a634-b15fda49ab56 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.407544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 38/72] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:46.408258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5fa1c-49fb-43d0-a634-b15fda49ab56 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.427608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.428296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.428584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.428861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.429653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:46.429968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.431399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.440316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 35/73] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.441419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.488292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.512078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5fa1c-49fb-43d0-a634-b15fda49ab56 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.541225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45e8be-b1c5-4f53-9baa-f20ca16e2d42 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.545814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.546612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.546897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.547177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.547936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:46.548245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.549706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c2b1e-8e5a-49fd-b9ca-1cad4c0b9fb5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.558143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 39/74] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.563183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.563475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.564068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.575408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.576054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.576342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.576621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.577410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:46.577992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.579394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039568a-e8cd-4101-b97f-a892e9be4da0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.581997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:46.583204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:46.587826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 36/75] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.589088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.589378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.589655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.605786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.605932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:46.607317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:46.654089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 40/76] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:46.657320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.680286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 37/77] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:46.680286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.727070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.770524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d42fe-8bcb-41d6-8ed1-2db85f241580 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.785705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335f727-6ddd-4742-8a60-fdd60f03daa9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.807775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.808673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.808986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.809267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.810496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.811948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.813871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 41/78] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.814722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.824563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.825471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.825781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.826041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.839504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.840990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.841112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.842681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 38/79] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.847489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.916422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b5bb-6702-4515-8554-b472ed612a36 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.938122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:46.943817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f5b18-5fe7-48c4-9fec-6d3a3a8e1d85 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:46.949921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.950811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.951100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.951388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.952628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.954317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.955988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 42/80] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.956755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:46.978796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:46.979596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:46.979877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:46.980149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:46.981333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:46.983182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:46.984850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 39/81] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:46.985621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.062913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.073543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de95c7-dc7f-4401-9f00-9385a34fa5cd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.090128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55731567-bde3-4e6a-9661-31cd153242b2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.108370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.109796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.110085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.110366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.120281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:47.123190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.123841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.124122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.124392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:47.128367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:47.129485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 43/82] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:47.130364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.160126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.198043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.199452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.232523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42370ed6-c41c-4e61-bbe0-df237cd8dcd0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.266553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.267204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.267487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.267762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.288436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.298643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 40/83] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:47.299488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.344561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.346182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.366732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.408724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5679-ea05-4867-bc8a-51351dd01a84 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.442194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.442854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.443140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.443417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.448849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 44/84] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:47.449928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.500539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.511294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.512788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.547074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3adb9-91c5-43bb-a2b7-55d753eca2e4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.572603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.580643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.581365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.581649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.581935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.582658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.584170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.609754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 45/85] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:50:47 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:50:47.610807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.612227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 41/86] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:47.613108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.731213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362638a-87fa-496e-8a41-285fef30aab3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.749295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b6717-5543-4f71-8703-e94bbd398d6b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.759727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.774843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.775891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.776187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.776461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.778271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.779187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.781088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 42/87] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:47.782604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.800224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.801385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.801743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.802022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.802791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:47.803103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.804566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.814287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 46/88] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:47.815343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:47.831228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:47.921478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284155-2a62-4f38-acc8-78b71bb3710a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.949552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39a027-db0d-4fe9-bbbc-6bb4f01670cc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:47.960646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:47.962075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:47.962487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:47.962914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:47.964231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:47.965865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:47.967839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 43/89] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:47.969645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.003400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.004210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.004490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.004765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.005996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.007473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.009402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 47/90] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:48.009978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.060094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:48.110408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:48.114990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b28f3-1bbe-48ce-9a7c-f1fdd08645ea None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.129934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc497a-087c-42fe-ba1f-8b1a82d2ce23 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.165236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.166279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.166752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.167014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.167449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.167899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.168180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.168454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.169190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=02f93a0d97354be4b96d139a23cac2ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:48.169654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.169901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.171122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.171837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.173495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 48/91] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:48 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:48.178244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.187547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 44/92] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:47 2022] GET /identity/v3/role_assignments?user.id=02f93a0d97354be4b96d139a23cac2ce&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:48.189239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.310371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a081579-3be0-4a35-b97c-472f7a564f4b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.354555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.356816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.361530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.362122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.363208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:48.363731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.365463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.385409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 45/93] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:48 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:48.386648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.424338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:48.471688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6cdaa-e6fb-4d9a-a0ec-45a39e08088f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.486495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:48.514262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2ef5-4071-4738-ad8a-630752dca603 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.557920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.559449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.560282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.561312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.562172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.563701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.565387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.565674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.566571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:48.567080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.569039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.586890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 49/94] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:48 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=724184bfe3d1497590814ff09b602192 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:48.660633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.666876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.702498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:48.790863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 46/95] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:48.798408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:48.952002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa0ea-ac74-4925-b327-25fe3b9b913d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:48.990932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:48.992167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:48.992635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:48.993160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] PATH_INFO: `/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:48.994565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdfd64d7e5843a18aee4ee630aecf13)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:48.996220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac4357-1062-47bf-ad03-487ef5989a16 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:48.998160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 50/96] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:48 2022] GET /identity/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:49.004827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:49.134632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860bcc7-371d-4d8a-b758-218c0646f2c4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:49.163343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:49.171609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:49.172425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:49.172670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:49.172999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:49.173821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=4cdfd64d7e5843a18aee4ee630aecf13, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:49.174137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:49.175620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a85d0-9b4d-4938-82dc-5b14db4f4897 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:49.181587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 47/97] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:49 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=4cdfd64d7e5843a18aee4ee630aecf13&role.id=724184bfe3d1497590814ff09b602192 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:50.344869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e92da-1214-4e3b-ac74-9133c1c66290 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.345407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e92da-1214-4e3b-ac74-9133c1c66290 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.345479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e92da-1214-4e3b-ac74-9133c1c66290 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.347004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 51/98] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:50.352251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.352655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.352822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.370098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:50.371307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:50.454991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 48/99] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:50.502706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e92da-1214-4e3b-ac74-9133c1c66290 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:50.556303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:50.636298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e92da-1214-4e3b-ac74-9133c1c66290 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:50.680301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:50.681158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.681449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.681726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.694800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ed9d7f5f-4225-4633-ab85-52b7bcfb7ac7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:50.702633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:50.704030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:50.704925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 52/100] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:50.711758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:50.810242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12785bb-22a8-4085-8e01-0f86dc4eeb9c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.810567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12785bb-22a8-4085-8e01-0f86dc4eeb9c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.810837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12785bb-22a8-4085-8e01-0f86dc4eeb9c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.812991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 53/101] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:50.820881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.821201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.821500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.841968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:50.843206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:50.862776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8814a-afd7-484b-adfd-32836cf9b572 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:50.909362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:50.929952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:50.930646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:50.930931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:50.931206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:50.931975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:50.932289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:50.938844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:50.951731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 49/102] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:50.957577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:50.984545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 54/103] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:50:51.022027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:51.062184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceffd-46b3-402e-9dbe-24629942f2df None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:51.069675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.097468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:51.098381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.098668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.098945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] PATH_INFO: `/v3/users/dc07252c43c34f148580b9f9edd88027` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.114115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=dc07252c43c34f148580b9f9edd88027)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:51.115624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:51.121042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 50/104] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:50 2022] GET /identity/v3/users/dc07252c43c34f148580b9f9edd88027 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:51.123823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:51.196014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da479df-22b7-4345-8b77-22f118743c8b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:51.213551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.279269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:51.280488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.281063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.281552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.283097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:51.284789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:51.286711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 55/105] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:51.295999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:51.333632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ee34b-0e5b-406c-a929-2ffa996fb3c5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:51.396271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:51.396997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.397815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.397815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.521746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.530986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78d969-d8cd-4d9d-be8b-c6d8d1f8b8ca None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:51.537468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:51.538991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e8fe6-92e3-497d-b1a8-fc6372b7822d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:51.574843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:51.575526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.575814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.576098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.576831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:51.578442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622a417-0e71-472b-a6ef-e4214cbeec6d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:51.593258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.607057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 56/106] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:50:51 2022] POST /identity/v3/projects => generated 302 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:50:51.607057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8b9-57c1-48ed-b4a9-5516557d1211 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.607057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8b9-57c1-48ed-b4a9-5516557d1211 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.607057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1b8b9-57c1-48ed-b4a9-5516557d1211 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.615272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 57/107] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:51.616635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440506-53f9-4ed4-a78e-65e0bf79901f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.616944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440506-53f9-4ed4-a78e-65e0bf79901f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.617224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440506-53f9-4ed4-a78e-65e0bf79901f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.618664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 58/108] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:51.624793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e966a-6f0b-40db-b7ac-3a3953d22cf3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.624793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e966a-6f0b-40db-b7ac-3a3953d22cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.625199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e966a-6f0b-40db-b7ac-3a3953d22cf3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.626618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 59/109] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:51.627650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5975b0-6712-486c-9fda-31844ae3903a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.628093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5975b0-6712-486c-9fda-31844ae3903a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.628370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5975b0-6712-486c-9fda-31844ae3903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.647668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0b5975b0-6712-486c-9fda-31844ae3903a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:51.649132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0b5975b0-6712-486c-9fda-31844ae3903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:51.673834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 51/110] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 28 11:50:51.675634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.676082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.676509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.697497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:51.698729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:51.757732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 60/111] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:51.758723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.759016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.759290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.796094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:51.798114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:51.804303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 52/112] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:51.805263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:51.892952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 61/113] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:51.892952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:51.903867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.908830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:51.951593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365620-0734-4aea-851e-2c00c977d84f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:51.989837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:51.990708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:51.990991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:51.991265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:51.992816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:51.994281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:51.996039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 53/114] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:51 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:51.996811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.031919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96a32b-2595-4b69-9599-23b8ee2f7384 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.081879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.082686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.083013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.083542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.098945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:52.109694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:52.110604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:52.111401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:52.112613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 62/115] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:50:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:52.112697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.123900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fe53d-a3fd-45c8-b567-92ee86f9c11e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.191499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.192797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.193552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.194029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.210067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.215654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:52.221493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:52.230438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:52.231862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:52.232846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:52.235133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 54/116] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:52.236219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.267588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c30d6-607b-44e7-8867-7147e445fdd2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.304699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.305777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.306229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.306656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.317517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:52.325434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:52.326470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:52.327404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:52.329873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 63/117] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:52.330867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.356260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb274e6c-6a9f-4912-bfef-dfd82cd64935 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.395821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.396333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.397094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.397094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.397722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=dc07252c43c34f148580b9f9edd88027) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:52.398034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.399548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.405310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 55/118] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/role_assignments?user.id=dc07252c43c34f148580b9f9edd88027&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.410234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.455315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da809d0-80c9-44c0-866a-3e06d51d09ab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.488262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.498771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.499907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.500423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.501259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.501773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.503182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:52.503696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.505522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000dda-9f12-4c04-95b7-252317749120 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.521369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 64/119] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.523313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749989-5737-4b17-8ccc-f82daadcb7b1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.523890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749989-5737-4b17-8ccc-f82daadcb7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.523890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749989-5737-4b17-8ccc-f82daadcb7b1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.525744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 65/120] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:52.526507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749989-5737-4b17-8ccc-f82daadcb7b1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.547602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b486cce-e2ae-4a4b-a733-07cc417cdb48 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.588892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.589264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.589908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.590017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.590804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:52.591109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.592625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.602845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 56/121] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.603980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.673242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749989-5737-4b17-8ccc-f82daadcb7b1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.722592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.723167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42f5bd-0c0a-4fae-9633-36a41322019a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.723914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.724401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.725061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.726875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:52.727430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.729109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95b22a-0710-4ee2-bcd0-aa2030c56d62 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.742059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 66/122] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.743808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.744321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.744802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.762258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.765362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:52.769761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:52.778975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.779870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.780158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.780431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.783953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.783953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.785130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 57/123] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.785746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.837350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:52.864817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 67/124] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:52.865995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.906928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a74e-f303-4bc3-845e-90a9bd115f76 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:52.941786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:52.943262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:52.943262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:52.943262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:52.944329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:52.945840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:52.947626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 58/125] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:52.948404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:52.975001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337333f3-52f4-4cb4-97f2-662d790a0af8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.013715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.014846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.015320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.015776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.016217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.018305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.020056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.022042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 68/126] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:53.023402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.068392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee5084-99aa-4922-a1eb-eec0f0fbbe6c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.088400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.113396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.114414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.114608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.114883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.116084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.117696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.124249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 59/127] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:52 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:53.125173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.151896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873355-2e73-4023-8f5e-89ad14faeb43 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.188723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.189807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.190230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.190644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.202127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:53.212587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:53.213684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:53.214591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:53.216638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 69/128] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:53.217718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.226975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f7e9b-66be-45ff-9b71-d44f6b63ccca None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.256675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.262654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.263767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.263889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.264083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.265592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.267617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.269756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 60/129] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:53 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:53.273852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.330886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e80b4-51f8-4b85-82b9-4f8b861a2b35 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.343970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.371875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.373059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.373482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.373990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.375549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.377248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.377719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d68ce-4d74-44fd-b4a1-1e01dc1a4a53 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.379800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 70/130] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:53 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:53.380827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.434477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.436679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.437242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.437836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.441545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7, role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.443219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.462239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 61/131] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:50:53 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/a0087dfad30d4fd095fbec15fc10286d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:50:53.463737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.540849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.595786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0adef-e3ab-46cd-bc0c-6d4fc333486b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.611429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6ba6e-cfc4-426f-a886-3b74adaf832e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.640445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.641890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.642363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.642817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.645759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.647470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.667716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 62/132] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:50:53 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:50:53.668933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.715274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.715886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.716157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.716439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.717242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:53.717559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.719100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.740430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 71/133] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:53.746178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:53.818723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5b51-edfb-4d1d-a0e3-89184270d243 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:53.840582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.863243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:53.865305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.865793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.866290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/02f93a0d97354be4b96d139a23cac2ce/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.868708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, user_id=02f93a0d97354be4b96d139a23cac2ce, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:53.870606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdadc72-0765-4068-9f5d-c9ef657f3bc6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:53.871771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:53.888732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 63/134] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:50:53 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/02f93a0d97354be4b96d139a23cac2ce/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:50:53.910904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acfa3-c2da-4c78-8160-551399a0a9ce None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.911694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acfa3-c2da-4c78-8160-551399a0a9ce None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.912237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197acfa3-c2da-4c78-8160-551399a0a9ce None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.914290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 64/135] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:53.923663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:53.924024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:53.924270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:53.958152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:53.959489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:53.983498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cb25b-41c9-4a5d-b800-6a77cc75fade None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:54.024189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:54.025655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.026232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.026769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.028321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:54.030177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:54.032348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 72/136] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:53 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:54.038165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:54.090581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 65/137] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:54.142598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:54.168866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90124c57-1bca-4115-a8c0-faecd80272a3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:54.178834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:54.213416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:54.214783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.215254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.215710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] PATH_INFO: `/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.217687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdfd64d7e5843a18aee4ee630aecf13)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:54.219726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:54.221554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 73/138] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:54 2022] GET /identity/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:54.234947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:54.319241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d880d-de44-4d61-b067-a2f4570042e5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:54.345308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:54.367666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:54.368580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.368862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.369174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.370363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:54.371841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:54.377013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 66/139] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:50:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:50:54.380132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:54.438962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3f585-b957-4975-9142-503fea25d005 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:54.492265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf69b5-8acb-446d-89f5-d4d9534934d6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:54.529525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:54.530224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.530510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.531044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.532279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:54.533800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9729ee9-31c0-46fa-8fdc-33f66e0743cf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:54.558314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:54.561649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.562010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.565188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] PATH_INFO: `/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.567598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdfd64d7e5843a18aee4ee630aecf13, user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:54.569939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd2159-5272-448b-b70b-924bc7f95e6e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:54.575078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 67/140] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:50:54 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:50:54.604862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 74/141] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:50:54 2022] PUT /identity/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13/users/1f6fbf1c7c8a43a0b25ed73d3762dff7/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:50:54.850254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0caf-0bc4-4fe5-ba6c-ffe0e12bcc3a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.850601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0caf-0bc4-4fe5-ba6c-ffe0e12bcc3a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.850818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c0caf-0bc4-4fe5-ba6c-ffe0e12bcc3a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.852294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 68/142] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:54.858322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:54.858818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:54.859049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:54.882058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:54.883303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:54.922378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:54.967240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 75/143] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:55.000081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0caf-0bc4-4fe5-ba6c-ffe0e12bcc3a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:55.040924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:55.134194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c0caf-0bc4-4fe5-ba6c-ffe0e12bcc3a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:55.185625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:55.186448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:55.186724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:55.187052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:55.202054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:55.210054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:55.210893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:55.211634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 69/144] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:55.219938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:55.372716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157f52-0f6d-482a-b126-9ff80863a26d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:55.422828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:55.423529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:55.423815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:55.424094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:55.424862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:55.425195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:55.426597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:55.440798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 76/145] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:55.446215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:55.587478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:55.596722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc97bc5-0ec5-453f-bff8-2f47d3557cf2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:55.633789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:55.634613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:55.634890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:55.635158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] PATH_INFO: `/v3/users/dc07252c43c34f148580b9f9edd88027` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:55.638189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] RBAC: Authorizing `identity:get_user(user_id=dc07252c43c34f148580b9f9edd88027)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:55.639673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:55.641747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 70/146] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:55 2022] GET /identity/v3/users/dc07252c43c34f148580b9f9edd88027 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:55.647636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:55.689031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:55.768451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90adb0-d595-4173-97fc-e6f8ff41f2b0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:55.806072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:55.806957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:55.807234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:55.807503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:55.811213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:55.813430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ede83-3ff5-492c-aaa1-fa996a73a2d9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:55.815416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 77/147] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:55 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:55.821268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:55.932360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560557-68ff-4b70-8930-7cec8683acf0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:55.967271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:55.968276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:55.968628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:55.969116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/dc07252c43c34f148580b9f9edd88027/roles/66b89bc3a380455493a8eda82c8404a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:55.971358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, user_id=dc07252c43c34f148580b9f9edd88027, role_id=66b89bc3a380455493a8eda82c8404a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:55.972828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c583aeb-113b-4778-a893-108cc81e5aea None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:55.992167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 71/148] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:50:55 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/users/dc07252c43c34f148580b9f9edd88027/roles/66b89bc3a380455493a8eda82c8404a3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:50:56.635578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69cc7-3fba-42a3-8d87-0c9b97fb3d36 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:56.636129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69cc7-3fba-42a3-8d87-0c9b97fb3d36 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:56.636129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69cc7-3fba-42a3-8d87-0c9b97fb3d36 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:56.637982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 78/149] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:56.649655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:56.650158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:56.650242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:56.671733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:56.673260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:56.688830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:56.763229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 72/150] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:56.805920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69cc7-3fba-42a3-8d87-0c9b97fb3d36 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:56.951443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:56.958127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69cc7-3fba-42a3-8d87-0c9b97fb3d36 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:57.004709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:57.005566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.005850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.006127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.024621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:57.032926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:57.033766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:50:57.034627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 79/151] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:50:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:57.042273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:57.181233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecde1a-5b2e-480a-a9b2-60879f987a16 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:57.224383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:57.225010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.225294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.225569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.226330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:57.226640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:57.228056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283cbae-f372-4491-8ba5-bcaa966f7619 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:57.239445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 73/152] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:50:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:57.245177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:57.334724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae757a-964f-4814-95b2-b48a7a5496ec None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.335062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae757a-964f-4814-95b2-b48a7a5496ec None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.335275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae757a-964f-4814-95b2-b48a7a5496ec None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.336815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 74/153] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:50:57.354884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe570-0376-4b37-9547-823427ce5ba2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.355197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe570-0376-4b37-9547-823427ce5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.355430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fe570-0376-4b37-9547-823427ce5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.382992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cf7fe570-0376-4b37-9547-823427ce5ba2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:57.384201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cf7fe570-0376-4b37-9547-823427ce5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:57.400010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:57.402922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998b73-dd3c-4bd5-9e8a-8ac95e59e8d9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:57.493718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:57.494533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.494817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.495093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.498796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:57.500353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3e989-8de6-47ea-8ea5-4c038b8cbbcd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:57.502243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 80/154] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:57 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:57.503343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574dccd-ca7e-4d5d-8160-398d6c8d51e4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.503629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574dccd-ca7e-4d5d-8160-398d6c8d51e4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.503904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574dccd-ca7e-4d5d-8160-398d6c8d51e4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.514859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 81/155] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:57.515798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574dccd-ca7e-4d5d-8160-398d6c8d51e4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:57.561925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 75/156] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:50:57.563399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.563685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.563963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.585299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:57.601752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:57.602964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:57.733163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574dccd-ca7e-4d5d-8160-398d6c8d51e4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:57.750056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 76/157] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:57.750878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:57.816959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:57.816959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:57.816959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:57.816959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:57.860665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:57.912938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e089-310d-4c45-9d3d-2507fed0affa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:57.940527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:57.946124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:58.037285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:58.038558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:58.039388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.039666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.039934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.073693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:58.086611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:58.087639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:58.088402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:58.095132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 77/158] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:58.096059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.137899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 82/159] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:58.138767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.321949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5413e6a-72b4-41b0-8b8c-dfd14994559f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:58.361074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:58.370069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:58.371078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.372174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.373590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.404289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-53f88c22-48f4-4dbc-a1d2-dae69aec4d6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:58.423891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:58.425044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:50:58.426018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:58.428821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 78/160] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:58.430810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490017-7277-4c12-9663-b9d7fbe606d8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.432997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49626af-52fb-4e7c-ac4f-9a3d07641219 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:58.434906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490017-7277-4c12-9663-b9d7fbe606d8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.436168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490017-7277-4c12-9663-b9d7fbe606d8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.437749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 79/161] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:58.441173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5490017-7277-4c12-9663-b9d7fbe606d8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.482480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:58.504265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:58.506375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.507940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.508356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.510149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:58.510598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:58.513213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:58.530737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 83/162] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:58.534007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.639734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5490017-7277-4c12-9663-b9d7fbe606d8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:58.692877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:58.693777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.694061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.694337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.696830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:58.698355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789201a1-0c07-4f2e-ad5b-40a3eb94b47c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:58.703845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4baf9-d31c-4825-b7b1-4669566c3391 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:58.705632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 80/163] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:58.706905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.707188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.707471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.735776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:50:58.737008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:50:58.758634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:58.759312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:58.759598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:58.759897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:58.760633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:58.760956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:58.762362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:58.785807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 84/164] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:58.786687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.836439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:58.852461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 81/165] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:50:58.854947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:58.886359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.011974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cddc56-1514-4d28-9a80-dfdd27f9f523 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.078973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d162f1-868d-42f5-bbd7-60d67420c290 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.085345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.095142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.095645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.096068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.097575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:59.099227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:59.101325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 85/166] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:59.104245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.166122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.167047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.167469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.167906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.172927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:50:59.173535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:59.175241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:59.185639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 82/167] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:50:58 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:59.186753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.337358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.365074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecc2c3-5a2e-43ab-bdb3-f45dbe6a6e10 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.399873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.433077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef7ae3-5866-43d6-b2a8-139fcb8bcb9d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.463029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.463937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.464215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.464488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] PATH_INFO: `/v3/users/02f93a0d97354be4b96d139a23cac2ce` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.469715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] RBAC: Authorizing `identity:get_user(user_id=02f93a0d97354be4b96d139a23cac2ce)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:59.471215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:59.473304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 86/168] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/users/02f93a0d97354be4b96d139a23cac2ce => generated 247 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:50:59.474092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.505422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.506257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.506538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.506810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.532587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:50:59.540778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:50:59.541832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:50:59.542598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:50:59.543840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 83/169] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:50:59.545095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.644410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d4197-9cb5-4758-a9d5-20499edc4963 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.679561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762bcbf-6297-466f-b101-b4d95765488b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:50:59.683664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.683972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.685549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.685549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.695457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.719934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:50:59.720597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.720880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.721183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.745424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.761747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:59.763235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6e9c3-cf00-4d74-8b53-e3de8ada7121 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:59.789193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:50:59.790693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6ec6-8867-48fa-8805-9e968551a57e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:50:59.868249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 87/170] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:59.869711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04a4f-04e5-45ed-a986-77d6d2b40a81 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.870001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04a4f-04e5-45ed-a986-77d6d2b40a81 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.870270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04a4f-04e5-45ed-a986-77d6d2b40a81 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.871560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 88/171] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:59.874391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd678-0330-45cb-86f5-857a4756ad89 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.874676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd678-0330-45cb-86f5-857a4756ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.874948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd678-0330-45cb-86f5-857a4756ad89 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.876729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 89/172] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:59.877049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd678-0330-45cb-86f5-857a4756ad89 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.893729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 84/173] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:50:59.895233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9153e8-c18e-4013-8e25-3dd02b7b3e2d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.895455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9153e8-c18e-4013-8e25-3dd02b7b3e2d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.895723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9153e8-c18e-4013-8e25-3dd02b7b3e2d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.897028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 85/174] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:59.901333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a39e5-19d3-45b8-a86e-7611daf461a3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:50:59.901616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a39e5-19d3-45b8-a86e-7611daf461a3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:50:59.901888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a39e5-19d3-45b8-a86e-7611daf461a3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:50:59.903108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 86/175] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:50:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:50:59.903794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a39e5-19d3-45b8-a86e-7611daf461a3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:50:59.919786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.968138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:50:59.987483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd678-0330-45cb-86f5-857a4756ad89 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.006772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a39e5-19d3-45b8-a86e-7611daf461a3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.022504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.023176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.023458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.023698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.024465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:00.024762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.026184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5daa40-eb1d-468e-a112-3cde61c5a3eb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.034315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 90/176] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.035487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.035751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.036021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.041668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.042552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.042832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.043083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.044269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.045741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4b1e5-a076-4f86-8a3e-cc6379cfbf0f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.047739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 87/177] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:50:59 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.048661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4e714-1211-4648-81a7-c7026999b581 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.048926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4e714-1211-4648-81a7-c7026999b581 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.049198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4e714-1211-4648-81a7-c7026999b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.054105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:00.055325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:00.065352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ffa4e714-1211-4648-81a7-c7026999b581 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:00.066513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ffa4e714-1211-4648-81a7-c7026999b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:00.121898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 91/178] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:00.129820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.130733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 88/179] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:00.142256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1942c2-1772-4740-90fd-207925a9fcf4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.142543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1942c2-1772-4740-90fd-207925a9fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.142809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1942c2-1772-4740-90fd-207925a9fcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.159223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2e1942c2-1772-4740-90fd-207925a9fcf4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:00.160319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2e1942c2-1772-4740-90fd-207925a9fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:00.167660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.210312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.229028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 89/180] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:00.230285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.230562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.230811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.233265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f232b-9b52-4880-9d9a-4536fd6eda16 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.252992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:00.254192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:00.268470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.269393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.269668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.269962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.271141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.272605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.274354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 92/181] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.279349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.319669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 90/182] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:00.320496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.376835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a2f0e-058e-4806-bfe5-26e9a97660d6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.389085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.412593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.413436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.413715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.413988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] PATH_INFO: `/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.415196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] RBAC: Authorizing `identity:get_user(user_id=1f6fbf1c7c8a43a0b25ed73d3762dff7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.416664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.417863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86668ad4-1d88-434b-b538-e1cfa44b913e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.418359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 93/183] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/users/1f6fbf1c7c8a43a0b25ed73d3762dff7 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.421788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.436310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.453771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.454443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.454803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.454977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.455809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:00.456117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.458233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.463382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 91/184] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=724184bfe3d1497590814ff09b602192 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.464062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.532697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c599bb8-fac9-413c-b2da-bf12f9d005e7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.570418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.571252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.571535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.571806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.583774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:00.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:00.592677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:00.593556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 94/185] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:00.594210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.609086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01964ff1-780c-4f32-bdec-114d8a0e8ed7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.651826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.652637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.652937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.653218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.656226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.666943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:00.679138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:00.680212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 92/186] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:00.681568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.683023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.714496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3eaf5-0dba-4483-927b-6e5ae8b4c844 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.747587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.748409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.748715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.749020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.759363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:00.767680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:00.770597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 95/187] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:00.772084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.790584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e50457-f565-48b4-a904-1a29cd6b3d76 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.823544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.824200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.824481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.824751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.825624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=02f93a0d97354be4b96d139a23cac2ce) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:00.825934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.827438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.833240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 93/188] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/role_assignments?user.id=02f93a0d97354be4b96d139a23cac2ce&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.834019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.882272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0d2aa-f701-402f-b81b-fb709540d4c5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.900480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.916138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:00.923590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.924664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.924971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.925247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:00.926446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:00.927933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:00.929662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 96/189] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:00.930534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:00.948835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f15a-b806-4fb4-bd25-b75814e1e6e2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:00.987951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:00.988624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:00.988933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:00.989374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.065153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420db4b-40a2-46fd-a7bb-e2a0642764ed None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.082354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.083898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.100809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.101511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.101789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.102058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.102801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:01.103102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.104534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.113204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 97/190] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.114009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.147162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.177201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.190774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 94/191] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:01.191608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.217642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152653-cd73-4709-92d9-a5fc2dced9e6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.252757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.253458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.253737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.254011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.254770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:01.255071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.256457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.264820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 98/192] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.266716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.289229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384e9b-bb03-4de6-aaac-a509a46b2fc1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.323256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.324148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.324423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.324695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.325485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:01.325788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.327161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.334915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 95/193] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.339542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.361344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.379739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dba994-f943-446d-a55a-4478eb004b9b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.413138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.424223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.424923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.425217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.425490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.426199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.427707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.438489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bbe7d-e592-45dc-94d9-dea37a8f5c5a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.447277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 99/194] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:01 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:01.448073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.490172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.491184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.491459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.491726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] PATH_INFO: `/v3/users/dc07252c43c34f148580b9f9edd88027` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.493021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] RBAC: Authorizing `identity:get_user(user_id=dc07252c43c34f148580b9f9edd88027)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.494534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.496558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 96/195] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/users/dc07252c43c34f148580b9f9edd88027 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.501961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.572871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b997441-3d2f-47e2-831f-d0d4aefefde6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.607058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.616732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.617013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.617235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.617779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] PATH_INFO: `/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.618805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] RBAC: Authorizing `identity:get_project(project_id=4cdfd64d7e5843a18aee4ee630aecf13)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.620100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2a98f-f982-4439-b88e-3ced34c83f7b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.620267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.626679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 100/196] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/projects/4cdfd64d7e5843a18aee4ee630aecf13 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.626679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.659623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.660543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.661185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.661185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.678983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.680391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.682236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 97/197] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.684964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.688204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.761528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ad67f-ee70-4f67-b0e4-860f2f64390c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.805718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.808054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.808762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.809283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.811388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.813920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.816092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 101/198] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.816348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38947c8-5f1e-4723-850e-8080382f9005 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.817279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:01.864520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.865361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.865854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.866241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.889358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.944363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36e366-25b9-44a6-8d44-4b37559e2df8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:01.948107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.950169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.961958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:01.983856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:01.984651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:01.984971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:01.985251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:01.986124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=4cdfd64d7e5843a18aee4ee630aecf13, user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:01.986574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:01.988700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:01.996233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 102/199] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/role_assignments?user.id=1f6fbf1c7c8a43a0b25ed73d3762dff7&scope.project.id=4cdfd64d7e5843a18aee4ee630aecf13&role.id=724184bfe3d1497590814ff09b602192 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:01.997500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.068186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 98/200] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:02.069591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.132197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236187c-4fc8-4afe-8909-94ab06014660 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.145298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.173299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.173982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.174267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.174543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.234267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.259360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.261193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.279094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29ed3-b47b-44de-802c-d5c4ebffb076 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.344849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.346106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.346584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.347064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.386643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 103/201] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:02.387815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.440986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.489651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.494247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.519710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850937-ea8a-4cff-bd40-dc7fddc487c5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.559089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 99/202] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:02.559778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.571066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.572771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.573285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.574216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.576051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.578467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.581833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 104/203] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:02.583270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.683784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9938dc9-1383-4c4d-bec0-e49006e2fc06 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.718953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dab46-9a1b-48cb-8253-692b5b6500da None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.729914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.731957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.732713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.733206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.734978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.737490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.739500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 100/204] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:02.740097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.760467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.766834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.767738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.768015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.768288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.773020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.774702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.777575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 105/205] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:02.778417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.811335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:02.864778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55205d7c-6e6f-4d95-82b6-f0cf26c68571 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.901226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.901891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.902175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.902450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.903292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f, user.id=dc07252c43c34f148580b9f9edd88027) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:02.903605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.903944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee4489-4dae-4189-ba77-67c81d1e0b73 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:02.905199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.911551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 101/206] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/role_assignments?user.id=dc07252c43c34f148580b9f9edd88027&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:02.912349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:02.950212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:02.951212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:02.951695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:02.952157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:02.955248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=e836d1f801dc454e8b53c4b201ee8393) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:02.958233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:02.960812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7037e5b-4419-4ed3-9bdd-e8639e7bb6a1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:02.974621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 106/207] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/role_assignments?user.id=e836d1f801dc454e8b53c4b201ee8393&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:03.048322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:03.072942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab56d4-11a1-40bb-acd7-901ff49938b9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:03.122867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:03.123949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:03.124588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:03.125183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:03.126275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=e836d1f801dc454e8b53c4b201ee8393) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:03.126769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:03.128506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251ffe2-084b-4b48-b19f-f02c313b48eb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:03.135224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 102/208] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:02 2022] GET /identity/v3/role_assignments?user.id=e836d1f801dc454e8b53c4b201ee8393&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:03.738559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340973-d6a9-4d9a-8d4f-d728369cb882 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:03.738973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340973-d6a9-4d9a-8d4f-d728369cb882 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:03.739148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340973-d6a9-4d9a-8d4f-d728369cb882 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:03.740727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 107/209] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:03.746333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:03.746622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:03.746899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:03.764327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:03.765805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:03.806705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d921b-2926-4dad-a264-354216752c53 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:03.807323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d921b-2926-4dad-a264-354216752c53 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:03.807817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d921b-2926-4dad-a264-354216752c53 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:03.809812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 108/210] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:03.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:03.816489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:03.816986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:03.834734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:03.836233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:03.839887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 103/211] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:03.861832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:03.905348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 109/212] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:03.929343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:03.941634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:03.949257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:03.975163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67695864-9b01-48e5-97ba-a89de6c3864d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.010532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.011623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.011895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.013331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:04.015058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:04.016946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 104/213] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:04.022369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:04.059139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5b571-db2f-4000-bd27-97070ab6dfdd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.099254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.100317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.100664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.101034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.115117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:04.125979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:04.126870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:04.127643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 110/214] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:04.134260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:04.153744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb759-aa8e-42da-a782-084d96727985 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.193026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.194109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.194561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.195026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.195913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:04.197811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:04.207320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:04.220225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:04.227535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 105/215] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:04 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:04.260637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1d1ef-6e96-4078-b75b-42ec4444b4cf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.298482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.299624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.300165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.300647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.301783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:04.302312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:04.303966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:04.315853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 111/216] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:04.322257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:04.437173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c749fe6-3b8e-46cb-9798-a05c282e8e9a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.471018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.472258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.472736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.473224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] PATH_INFO: `/v3/users/857805d936b94f039cee06845f304cdb` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.487691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] RBAC: Authorizing `identity:get_user(user_id=857805d936b94f039cee06845f304cdb)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:04.489394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:04.491613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 106/217] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:04 2022] GET /identity/v3/users/857805d936b94f039cee06845f304cdb => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:04.497274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:04.603850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be45517-4fb7-444f-b496-8d8b7f43ff9b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:04.636710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:04.642623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:04.643290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:04.643573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:04.643846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:04.716297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:04.717867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:04.848593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 112/218] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:04.854195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:04.944880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:05.003024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70d5c6-ab7c-4a4f-8e81-b9f647d69f4e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:05.044803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:05.045696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.045975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.046437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.047748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:05.049242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edd311-d13d-480e-87fd-9fbbbeb969e9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:05.050994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 107/219] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:04 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:05.056851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:05.089961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:05.160657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1dd1-3b27-45dd-a590-df73b6d83e43 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:05.204082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:05.204764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.205081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.205357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.206207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=857805d936b94f039cee06845f304cdb) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:05.206518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:05.207982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf77949-c986-4dfa-ab29-76413613c2cb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:05.221078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 113/220] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/role_assignments?user.id=857805d936b94f039cee06845f304cdb&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=724184bfe3d1497590814ff09b602192 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:05.262035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d8be-111b-40bb-9109-abb8b1808e81 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.262591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d8be-111b-40bb-9109-abb8b1808e81 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.262591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09d8be-111b-40bb-9109-abb8b1808e81 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.264220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 108/221] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:05.270169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409c416-8687-4bb8-940d-7dcafd208d8b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.270458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409c416-8687-4bb8-940d-7dcafd208d8b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.270741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409c416-8687-4bb8-940d-7dcafd208d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.294154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e409c416-8687-4bb8-940d-7dcafd208d8b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:05.295421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e409c416-8687-4bb8-940d-7dcafd208d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:05.399081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 114/222] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:05.425187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09d8be-111b-40bb-9109-abb8b1808e81 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:05.459720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d9e9-b9d4-4130-bf67-00ad4495b957 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.460044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d9e9-b9d4-4130-bf67-00ad4495b957 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.460275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43d9e9-b9d4-4130-bf67-00ad4495b957 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.461916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 115/223] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:05.467302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.467587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.467865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.497072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:05.498292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:05.541271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:05.587751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09d8be-111b-40bb-9109-abb8b1808e81 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:05.620491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:05.637569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:05.638596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.640312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.641770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.655491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 116/224] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:05.660926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:05.668985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:05.670027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:05.671004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:05.673033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 109/225] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:05.682062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:05.705618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:05.806220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d241a-7a0e-42b3-a75b-ed4e7234f4b4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:05.846453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:05.847145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.847423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.847697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.848457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:05.848762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:05.850203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:05.875278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 117/226] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:05.893653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:05.903377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19dffa-e293-44d7-b074-fe01f7a80f35 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:05.941161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:05.941362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:05.941907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:05.942187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:05.942458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:05.954795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:05.963399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:05.964278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:05.965152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 110/227] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:05.972236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.029934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:06.112172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241346d2-9ea2-4876-bce8-cbba5b09f49b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.135524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d18d1-c5a9-4272-b09e-eb3b4e02d2e3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.168959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.170253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.170731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.170926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.171449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.171634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.171918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.172186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.176925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:06.176925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.176925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.177991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.179719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.181948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 118/228] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:06.184309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 111/229] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:06.188136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.189088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.296957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de84ed-e20c-4431-a3a2-bf18fafdccdb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.325327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:06.337607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.338918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.339452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:06.339977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.340143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.341543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.343242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.357469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 112/230] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:06 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:06.359445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d73769-0ecf-4859-a0ca-5ca76d8c9330 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.365945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.395258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.396278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.396786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.396957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.398325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.399920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.402166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 119/231] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:06 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:06.410129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.519758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcb1ef-a0d4-46db-b115-c14064e19690 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.574988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.575790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.576066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.576334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.578640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.580118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.584538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 113/232] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:06 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:06.589722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:06.617844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0bd5d-b19b-4a42-8424-9f29a8ccbbdb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.666415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:06.687228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.688188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.688470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.688740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/users/e836d1f801dc454e8b53c4b201ee8393/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.692216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, user_id=e836d1f801dc454e8b53c4b201ee8393, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.693775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fc240-8853-4891-9735-51bd856d55c0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.717849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 120/233] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:06 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/users/e836d1f801dc454e8b53c4b201ee8393/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:06.801448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3c928-3a93-4544-8117-e4c7d54b837d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:06.871132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:06.871980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.872264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.872542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/users/e836d1f801dc454e8b53c4b201ee8393/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.875258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, user_id=e836d1f801dc454e8b53c4b201ee8393, role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:06.876770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b322e6-6366-4def-ad0b-fb72f966f35e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:06.903612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 114/234] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:06 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/users/e836d1f801dc454e8b53c4b201ee8393/roles/a0087dfad30d4fd095fbec15fc10286d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:51:06.938684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb749cd-dc05-41b9-9fb4-451c97603d4d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.939201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb749cd-dc05-41b9-9fb4-451c97603d4d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.939276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb749cd-dc05-41b9-9fb4-451c97603d4d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.940642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 121/235] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:06.947168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:06.947474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:06.947739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:06.967609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:06.968863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:07.044264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5953-7a66-4898-b502-0cca02943a42 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.044578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5953-7a66-4898-b502-0cca02943a42 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.044826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5953-7a66-4898-b502-0cca02943a42 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.046938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 122/236] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:07.047912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 115/237] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:07.052666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.052983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.053264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.071236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:07.072521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:07.073700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.083676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.132675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.146876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 123/238] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:07.171587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.239692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a0c5b-9ab6-49ff-9e57-f76a96262e9e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.285662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.285868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabba83-4e8f-40c5-b069-173d1aed6284 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.286528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.286813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.287226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.289270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:07.290762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:07.292746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 116/239] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:07.298808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.340340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.341172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.341450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.341716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.364389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:07.374040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:07.375008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:07.375882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:07.376485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 124/240] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:07.390201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.409303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.506924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.509614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16ef14-4d79-4c06-b060-8ab60299fb07 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.545873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.546937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.547225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.547508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.548232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:07.549742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638de45-674d-4590-9945-b423c2cc869d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:07.567982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746512e-4626-4feb-a9ad-9f7eb024bf06 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.576972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 117/241] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:07 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:51:07.610612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611371c1-a436-47a3-89ba-181daf4d6db5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.610925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611371c1-a436-47a3-89ba-181daf4d6db5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.611171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611371c1-a436-47a3-89ba-181daf4d6db5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.612575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 118/242] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:07.618611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.618900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.619160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.621954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.622668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.622953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.623230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.624022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:07.624523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:07.626481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:07.641084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 125/243] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:07.642827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.658291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:07.660131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:07.727040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.778609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5540c-4758-4ead-818a-dbca1bdb7620 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.816687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.817644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.818002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.818280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] PATH_INFO: `/v3/users/ca3881d93eb2460f8f2090a1d0738799` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:07.835624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3881d93eb2460f8f2090a1d0738799)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:07.837261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:07.839306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 126/244] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/users/ca3881d93eb2460f8f2090a1d0738799 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:07.844866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.851186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 119/245] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:07.878137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:07.955346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a01cd-cdf3-4de6-bea5-c2f649920d06 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.970536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.982858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:07.990043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19c80f-4031-4123-b03c-5746a1a8682b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:07.994566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:07.995265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:07.995760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:07.996216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.027706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.028522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.028806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.029128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.040116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:08.051834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:08.052787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:08.053676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:08.057029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 120/246] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:08.059757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.071511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.073048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.174307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011aa9c-87e5-4803-9684-a59cca0eed10 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.207872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 127/247] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 28 11:51:08.213447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.215568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.216432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.216896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.217287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.218186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:08.218647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.220145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.235428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 121/248] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:08.236649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.241547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:08.265272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:08.359581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b4017-9ea5-425a-bfd8-6005a504c0eb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.374003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a4fd3-61ab-408d-a906-19ef004437bf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.407510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.409056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.409347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.409880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] PATH_INFO: `/v3/users/857805d936b94f039cee06845f304cdb` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.411121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=857805d936b94f039cee06845f304cdb)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.412604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.414343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 122/249] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:08 2022] GET /identity/v3/users/857805d936b94f039cee06845f304cdb => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:08.415199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.416154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.416432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.416702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.417984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.420350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.420467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.422658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 128/250] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:08 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:08.430485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.527040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:08.553105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1dbc-4fef-4974-abd1-f3e490aa0cc5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.598715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.599613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.599896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.600171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.601453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.605330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.607476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 123/251] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:08 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:08.614940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:08.637181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:08.652627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebb1a1-af06-49d3-b402-ab24cbf1c319 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.726371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.727398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.727873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.728365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.729444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=ca3881d93eb2460f8f2090a1d0738799) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:08.729970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.731988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288c6cd-6596-4557-8427-8f17d6faa9df None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.738661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 129/252] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:08 2022] GET /identity/v3/role_assignments?user.id=ca3881d93eb2460f8f2090a1d0738799&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:08.794089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278912c8-39e5-4096-b43d-871a7c025fa9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:08.846879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:08.847814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:08.848099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:08.848379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/users/857805d936b94f039cee06845f304cdb/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:08.854666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, user_id=857805d936b94f039cee06845f304cdb, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:08.856556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e7417-a88e-40cc-9676-d8cb7721c73b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:08.880054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 124/253] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:08 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/users/857805d936b94f039cee06845f304cdb/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:09.326181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69127dd-9039-48dc-bb91-db36572f68b9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:09.327217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69127dd-9039-48dc-bb91-db36572f68b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:09.327729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69127dd-9039-48dc-bb91-db36572f68b9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:09.329634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 130/254] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:09.335405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:09.335925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:09.336414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:09.342906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:09.357028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:09.358527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:09.439817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 125/255] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:09.477262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69127dd-9039-48dc-bb91-db36572f68b9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:09.611693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69127dd-9039-48dc-bb91-db36572f68b9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:09.648643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:09.649478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:09.649757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:09.650076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:09.661246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:09.668857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:09.669751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:09.670568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 131/256] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:09.678147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:09.814470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaee4aa-c9b7-4813-bf1f-5cb239c6d63b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:09.860518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:09.861246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:09.861529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:09.861857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:09.862575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:09.862881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:09.872410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd6f12-ce61-4fd5-ace0-b2b0cc967ee8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:09.872410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:09.881673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 126/257] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:09.891356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:09.920941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:10.018971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe9e1-8d83-4dd9-937d-6a3da0e741f5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.019646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe9e1-8d83-4dd9-937d-6a3da0e741f5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.020158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe9e1-8d83-4dd9-937d-6a3da0e741f5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.022062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 127/258] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:10.027011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.027510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.027974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.033935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e9a4-b48d-483e-8090-8b90b1fc5985 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:10.060828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:10.062542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:10.081539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:10.082443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.082725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.083012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.084200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:10.085696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:10.093987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 132/259] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:09 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:10.105834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.194904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 128/260] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:10.241344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:10.246274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.270248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea29a1-e2f0-48fe-8d51-826666870d10 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:10.321469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:10.322122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.322406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.322681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.409188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741cd5-696e-475d-bd7c-24b92da47dc1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:10.453442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:10.468265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:10.470547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:10.473275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:10.474224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.474508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.474773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.501851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:10.510235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:10.511094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:10.511878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:10.517366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 129/261] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:10.522771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.580874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:10.598686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 133/262] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:10.605449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.703431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac49c6c-4186-4628-aebc-787c09e23115 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:10.776444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4711e-1eef-460b-86fa-81d545d075cc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:10.837637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:10.838339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.838622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.838891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.839657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:10.839964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:10.841119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:10.841441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:10.842018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:10.842298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:10.842570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:10.849739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:10.851230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:10.853343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 134/263] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:10.860300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.867330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 130/264] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:10.872654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:10.939546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:10.990127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.018237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9b94e-b59c-40b8-8187-ee557888c68e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.069699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0408-6142-4540-b023-53079b15c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.070306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0408-6142-4540-b023-53079b15c4da None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.070595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0408-6142-4540-b023-53079b15c4da None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.070860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0408-6142-4540-b023-53079b15c4da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.071699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e0408-6142-4540-b023-53079b15c4da None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=e836d1f801dc454e8b53c4b201ee8393) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:11.072006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e0408-6142-4540-b023-53079b15c4da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:11.073496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e0408-6142-4540-b023-53079b15c4da None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:11.084851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 135/265] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/role_assignments?user.id=e836d1f801dc454e8b53c4b201ee8393&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:11.096808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62771233-c054-4846-ab4f-e4aaeaa9c14d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.136825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.137827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.138320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.138582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] PATH_INFO: `/v3/users/e836d1f801dc454e8b53c4b201ee8393` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.140017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=e836d1f801dc454e8b53c4b201ee8393)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:11.142343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dabaed-db38-462d-b7f0-503264cbfdd4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:11.144361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 131/266] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:10 2022] GET /identity/v3/users/e836d1f801dc454e8b53c4b201ee8393 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:11.149918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0408-6142-4540-b023-53079b15c4da None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.283529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0408-6142-4540-b023-53079b15c4da None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.298633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edef31-e871-42ea-81e3-270a18a7a167 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.298992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edef31-e871-42ea-81e3-270a18a7a167 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.299179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edef31-e871-42ea-81e3-270a18a7a167 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.300775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 132/267] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:11.308402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.308402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.308402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.308714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.325363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:11.326880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:11.335663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.336506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.336684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.336950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.405011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 133/268] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:11.425148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.430343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:11.432032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:11.505311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.544492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 136/269] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:11.569340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.576513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a7fca-2575-4649-b689-71d716c94bfe None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.597004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.617685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.618888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.620381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.621482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.644407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:11.654974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:11.656229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:11.657103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:11.663395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 134/270] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:11.666179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.764630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45d0eb-98f6-48f3-9038-213e0c1b068f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.810595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b1a0e-138d-4d5c-80ff-1e5593a5614e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:11.820510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.836421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.837318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.837600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.837871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.839199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:11.840943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85103327-8c57-4957-a0a0-7aefaa70e5b8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:11.843254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 137/271] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:11.844757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250cdc-801f-427b-a6dc-a9a4814667f2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.845217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250cdc-801f-427b-a6dc-a9a4814667f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.845646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250cdc-801f-427b-a6dc-a9a4814667f2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.847077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 138/272] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:11.852509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911aef-28a9-4b3e-bde5-aabf0a77da70 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.853028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911aef-28a9-4b3e-bde5-aabf0a77da70 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.853481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911aef-28a9-4b3e-bde5-aabf0a77da70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.856817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:11.857685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.857969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.858239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.859064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:11.859367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:11.860768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:11.869506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 135/273] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:11.870535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.871504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bb911aef-28a9-4b3e-bde5-aabf0a77da70 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:11.872976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bb911aef-28a9-4b3e-bde5-aabf0a77da70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:11.941314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 139/274] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:11.949079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314399-75fe-4df0-9e81-56e82d360f6f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:11.949079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314399-75fe-4df0-9e81-56e82d360f6f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:11.949079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28314399-75fe-4df0-9e81-56e82d360f6f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:11.949079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 140/275] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:11.949079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28314399-75fe-4df0-9e81-56e82d360f6f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:11.970447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:11.979470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b1f22-6062-43bc-822c-8dd4db8ac4ff None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.014940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.015648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.015926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.016194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.017055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=e836d1f801dc454e8b53c4b201ee8393) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:12.017387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.019078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee34e-cacf-4875-b2ef-b4441d4b24f4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.028269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 136/276] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/role_assignments?user.id=e836d1f801dc454e8b53c4b201ee8393&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.028269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09886-6cc5-4b30-8e19-361c3e767b54 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.028269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09886-6cc5-4b30-8e19-361c3e767b54 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.028269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed09886-6cc5-4b30-8e19-361c3e767b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.052234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6ed09886-6cc5-4b30-8e19-361c3e767b54 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:12.053494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6ed09886-6cc5-4b30-8e19-361c3e767b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:12.069666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28314399-75fe-4df0-9e81-56e82d360f6f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.083318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.115395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.116197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.116484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.116740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] PATH_INFO: `/v3/users/c95477af870f4dada1013a9a9b90bed1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.132086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] RBAC: Authorizing `identity:get_user(user_id=c95477af870f4dada1013a9a9b90bed1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.133962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.135931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 141/278] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:11 2022] GET /identity/v3/users/c95477af870f4dada1013a9a9b90bed1 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.136890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.140895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 137/278] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:12.142603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ab91a-ad55-479b-80e4-6d2ca11dc13b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.142894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ab91a-ad55-479b-80e4-6d2ca11dc13b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.143171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ab91a-ad55-479b-80e4-6d2ca11dc13b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.144610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 138/279] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:12.148379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ab91a-ad55-479b-80e4-6d2ca11dc13b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.230126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.265792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff486e-684e-4cd6-bf48-0f7d1387383c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.273182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ab91a-ad55-479b-80e4-6d2ca11dc13b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.301610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.302413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.302691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.302958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.304118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.305611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea972bd1-c854-4307-bbce-c1d1fb9f9c80 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.307248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 142/280] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.308437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.308597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.308718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.309011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.309282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.309558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.309827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.310647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, user.id=c95477af870f4dada1013a9a9b90bed1, scope.system=all) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:12.310954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.313150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.318395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 139/281] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/role_assignments?user.id=c95477af870f4dada1013a9a9b90bed1&scope.system=all&role.id=724184bfe3d1497590814ff09b602192 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.319179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.337043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:12.338272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:12.355718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.422932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 143/282] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:12.424321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.451848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b1379-8cb5-4eb8-8f6a-62500127c66f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.489596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.490719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.491066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.491401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.502136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:12.509842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:12.511163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:12.511968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 140/283] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:12.516984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.517319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.540998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ea683-b3f3-437c-8a97-015546688338 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.577787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.578564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.578915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.579194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.579920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.581448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.603979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.606356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 144/284] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:12 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:12.613107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.641126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1cc4-9812-4157-ac80-a66b06c177f9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.692096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.693048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.693433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.693807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.709256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:12.720415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:12.721578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:12.722521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 141/285] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:12.729445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.737290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed091573-bed5-4169-b07c-bbdadf3f4841 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.774362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.787760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.788459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.788745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.789065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.790175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:12.790482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.791899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.803750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 145/286] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.808775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.863067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37901439-5da7-4eaf-a556-f94f131dff0b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.911677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.912599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.913101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.913487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.914342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:12.914741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.916231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:12.925335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 142/287] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:12.933453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:12.943761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5810f1-4b57-4c3b-bebc-8014c75dff22 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:12.946692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:12.994157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:12.995062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:12.995337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:12.995608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] PATH_INFO: `/v3/users/ca3881d93eb2460f8f2090a1d0738799` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:12.996995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3881d93eb2460f8f2090a1d0738799)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:12.998480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.000567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 146/288] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/users/ca3881d93eb2460f8f2090a1d0738799 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:13.006081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:13.064056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9077b3-5d1e-4096-8567-2bf579b40911 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.081391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:13.113761 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.114836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.115119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.115701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] PATH_INFO: `/v3/users/857805d936b94f039cee06845f304cdb` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.116971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=857805d936b94f039cee06845f304cdb)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.118871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.121000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 143/289] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:12 2022] GET /identity/v3/users/857805d936b94f039cee06845f304cdb => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:13.127851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:13.141549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6787d-4299-40d4-a368-459f89f07d51 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.176268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.177197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.177471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.177736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.178915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.180364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.182481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 147/290] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:13 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:13.189927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:13.218766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:13.243298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394180-b835-45ee-85a5-e91fb91dcfb9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.286598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.287372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.289229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.289622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.372133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7114650-23c2-4cfe-8828-ac17968bd3e4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.413257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:13.418013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.419585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.453709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.454630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.454910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.455183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/users/ca3881d93eb2460f8f2090a1d0738799/roles/66b89bc3a380455493a8eda82c8404a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.461699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, user_id=ca3881d93eb2460f8f2090a1d0738799, role_id=66b89bc3a380455493a8eda82c8404a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.463171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.493649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 148/291] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:13 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/users/ca3881d93eb2460f8f2090a1d0738799/roles/66b89bc3a380455493a8eda82c8404a3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:13.507715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:13.543149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 144/292] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:13.549403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:13.675227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3cf0-5f6a-48f6-9611-4c4d2682bf05 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.710847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.711780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.712059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.712325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.713551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.715020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531ebe8-9509-45db-b5ad-0ea0477b69f7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.717187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 149/293] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:13 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:13.723356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:13.854473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5fae1-b0a5-4500-84e8-bf8365fb1108 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:13.890310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:13.891331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:13.891802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:13.892261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:13.893312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=857805d936b94f039cee06845f304cdb) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:13.893867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:13.895515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efff692-b97b-4379-91d4-afa6f8c5432c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:13.902209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 145/294] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:13 2022] GET /identity/v3/role_assignments?user.id=857805d936b94f039cee06845f304cdb&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=724184bfe3d1497590814ff09b602192 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:14.542814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345188fb-0e64-4a6b-bced-dfd4bd69d46e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:14.543218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345188fb-0e64-4a6b-bced-dfd4bd69d46e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:14.543401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345188fb-0e64-4a6b-bced-dfd4bd69d46e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:14.545043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 150/295] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:14.551589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:14.552036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:14.552208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:14.569973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:14.571161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:14.614748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:14.656080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 146/296] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:14.679005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345188fb-0e64-4a6b-bced-dfd4bd69d46e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:14.736715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:14.791379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345188fb-0e64-4a6b-bced-dfd4bd69d46e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:14.825174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:14.825899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:14.826175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:14.826446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:14.837431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:14.845254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:14.846128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:14.846897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 151/297] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:14.854135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:14.963129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79b5cc-ac32-454e-82bc-ac89d4ba2ca5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.000379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.001455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.001923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.002398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.003329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:15.003820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.005618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b49887-bc51-4272-97cd-a36fe9371744 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.014943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 147/298] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:15.021257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.066179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44da4-9830-4d9b-921b-e61e9460fb9e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.066804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44da4-9830-4d9b-921b-e61e9460fb9e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.067266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44da4-9830-4d9b-921b-e61e9460fb9e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.069115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 148/299] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:15.073662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.074230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.074697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.094069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:15.095601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:15.124550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031ba3-0872-451f-9a4b-47024bb24f1a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.126367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.137940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.159609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.160769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.160877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.161154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] PATH_INFO: `/v3/users/c95477af870f4dada1013a9a9b90bed1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.162338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] RBAC: Authorizing `identity:get_user(user_id=c95477af870f4dada1013a9a9b90bed1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.163891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.170408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 152/300] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/users/c95477af870f4dada1013a9a9b90bed1 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:15.171682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.184174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 149/301] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:15.207570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.306613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87895690-1e3e-4426-986f-fb0f980d702a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.315656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b87c8-22f8-4196-bf60-d694649dc7de None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.342611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.343709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.344150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.344431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] PATH_INFO: `/v3/system/users/c95477af870f4dada1013a9a9b90bed1/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.346192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c95477af870f4dada1013a9a9b90bed1, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.347615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.364963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 153/302] 10.222.0.55 () {58 vars in 1334 bytes} [Tue Jun 28 11:51:15 2022] PUT /identity/v3/system/users/c95477af870f4dada1013a9a9b90bed1/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:51:15.373915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.373915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.373915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.373915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.385631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-fb0f190e-4dcb-45dc-b196-8d4028c75d30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:15.394775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:15.395748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:15.396746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 150/303] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:15.402652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.424116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13247c9d-1bcd-4c58-b990-539aad569c7c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.424437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13247c9d-1bcd-4c58-b990-539aad569c7c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.424673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13247c9d-1bcd-4c58-b990-539aad569c7c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.426543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 151/304] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:15.431910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.432253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.432461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.456736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:15.458377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:15.469390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.482933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.521863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a2c2-aedf-4910-8674-fdd82f4d1e9d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.559786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 152/305] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:15.566447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.567129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.567411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.567685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.568441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:15.568748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.570169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.578729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 154/306] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:15.585626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.585785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.696748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72de0-9873-4faa-98fc-65595357525a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.710546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95744-b0a6-465f-9212-5d01fa2a4762 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.725309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.736940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.738629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.739031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.739402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] PATH_INFO: `/v3/users/794ba44b48ce47f481df634e72172377` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.753476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.754534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.754953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.755359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.756743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.757984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] RBAC: Authorizing `identity:get_user(user_id=794ba44b48ce47f481df634e72172377)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.758696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.759566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.765617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 155/308] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:15.766202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 153/308] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/users/794ba44b48ce47f481df634e72172377 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:15.767107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.767338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:15.784425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:15.883299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe6c92-add2-4219-ac9d-b0544dd76ef4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.889907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105b02-e1dc-4b40-b709-4eba87a3581d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:15.920977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.920977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.921374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.921938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] PATH_INFO: `/v3/groups` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.922339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.923808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f949c4c-ab35-4751-8430-75897ee28171 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.926586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:15.927254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.927536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.927794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.928641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, user.id=794ba44b48ce47f481df634e72172377, scope.system=all) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:15.928977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:15.931164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1606b4-ccec-4c5b-be98-091f627769e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:15.937601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 154/309] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Jun 28 11:51:15 2022] GET /identity/v3/role_assignments?user.id=794ba44b48ce47f481df634e72172377&scope.system=all&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:15.940174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 156/310] 10.222.0.55 () {60 vars in 1134 bytes} [Tue Jun 28 11:51:15 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:15.950703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf434d-a150-413b-a090-e67be3e4b858 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.951359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf434d-a150-413b-a090-e67be3e4b858 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.951827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf434d-a150-413b-a090-e67be3e4b858 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.958628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 155/311] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:15 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:15.961090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:15.961692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:15.962189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:15.983012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:15.984873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:16.019807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:16.081113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 157/312] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:16.100528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf434d-a150-413b-a090-e67be3e4b858 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:16.183984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:16.216313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf434d-a150-413b-a090-e67be3e4b858 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:16.251193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:16.252003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:16.252274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:16.252541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:16.263200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:16.271215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:16.272108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:16.272874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:16.277026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 156/313] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:16.278705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:16.379650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c54a7-cfc1-44d9-9416-b8e74034be57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:16.414457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:16.415147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:16.415425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:16.415697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:16.416450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:16.416754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:16.418183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:16.431778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 158/314] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:16.433158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:16.539859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4e7d8-e00c-41bf-8a43-e7ce3735b530 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:16.576375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:16.578268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:16.578724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:16.579194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] PATH_INFO: `/v3/users/ca3881d93eb2460f8f2090a1d0738799` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:16.580721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] RBAC: Authorizing `identity:get_user(user_id=ca3881d93eb2460f8f2090a1d0738799)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:16.582518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:16.584951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 157/315] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:16 2022] GET /identity/v3/users/ca3881d93eb2460f8f2090a1d0738799 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:16.590567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:16.625300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:16.693710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a60ef-7b31-461e-a58a-17b4fc2eef29 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:16.727165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:16.727796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:16.728082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:16.728351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:16.796514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:16.797937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:16.829633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:16.902325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 159/316] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:16.905628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:16.921679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:17.009595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a30e7-c506-41f7-ab6b-729ac10e9484 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:17.049120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:17.050348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:17.050835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:17.051306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:17.052710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:17.054377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f503a3-4cf4-474d-96b0-c5c6fde0e247 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:17.056592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 158/317] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:16 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:17.062684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:17.168416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73c7e7-c7c9-48d6-b463-efc3019f0a94 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:17.195813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:17.205478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:17.206348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:17.206866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:17.207138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:17.208054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, scope.project.id=c837898897aa468aa0e08029f92081cd, user.id=ca3881d93eb2460f8f2090a1d0738799) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:17.208365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:17.209916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b247f5-a740-40a6-9e2c-33bb64ba7c82 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:17.217317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 160/318] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:17 2022] GET /identity/v3/role_assignments?user.id=ca3881d93eb2460f8f2090a1d0738799&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:17.564845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1aabe-41d7-462b-80a1-9ecaedbd4e2c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:17.565287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1aabe-41d7-462b-80a1-9ecaedbd4e2c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:17.565387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1aabe-41d7-462b-80a1-9ecaedbd4e2c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:17.566987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 159/319] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:17.572096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:17.572494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:17.572843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:17.590888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:17.592112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:17.666196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 161/320] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:17.690527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1aabe-41d7-462b-80a1-9ecaedbd4e2c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:17.774998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:17.811743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1aabe-41d7-462b-80a1-9ecaedbd4e2c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:17.848483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:17.849229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:17.849556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:17.849827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:17.860938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:17.868711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:17.869728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:17.870520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 160/321] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:17.876104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:17.985861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea3817-8ed6-4472-ace0-09f2ed073abf None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.007808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.024356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.025067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.025325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.025609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.026361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:18.026660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.028055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5851e-a0b9-4db3-8349-389def80c436 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.037742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 162/322] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:18.043615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.056141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f55bb-119f-47c8-aac1-bc6dcb2bcbf3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.056430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f55bb-119f-47c8-aac1-bc6dcb2bcbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.056695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f55bb-119f-47c8-aac1-bc6dcb2bcbf3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.058583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 163/323] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:18.064128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220414-0cf5-402d-a461-7f2300714c26 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.064393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220414-0cf5-402d-a461-7f2300714c26 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.064668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220414-0cf5-402d-a461-7f2300714c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.082375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-03220414-0cf5-402d-a461-7f2300714c26 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:18.083546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-03220414-0cf5-402d-a461-7f2300714c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:18.150303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8d09a-0f3a-49e3-88a8-c39c92ae104c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.153943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 164/324] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:18.155384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3bbb-459c-46db-80df-d91414c512ad None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.156182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3bbb-459c-46db-80df-d91414c512ad None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.156461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca3bbb-459c-46db-80df-d91414c512ad None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.157760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 165/325] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:18.166300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.166577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.166846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.179647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.183649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:18.184831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:18.185814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.186897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.187178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.187449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] PATH_INFO: `/v3/users/c95477af870f4dada1013a9a9b90bed1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.188640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] RBAC: Authorizing `identity:get_user(user_id=c95477af870f4dada1013a9a9b90bed1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.190123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.191775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 161/326] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/users/c95477af870f4dada1013a9a9b90bed1 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:18.192541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.252616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 166/327] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:18.253431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.297556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.298903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e92a22-9fdd-4b6c-9cf7-1ba54265a3bf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.332357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.333201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.333486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.333751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.344487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:18.352269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:18.353174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:18.353970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:18.354559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1f34b-6261-4cfa-8868-df38983fb33f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.354747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 162/328] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:18.354747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.389837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.390493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.390776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.391048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.391877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] RBAC: Adding query filter params (role.id=724184bfe3d1497590814ff09b602192, user.id=c95477af870f4dada1013a9a9b90bed1, scope.system=all) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:18.392184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.393657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.417053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 167/329] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/role_assignments?user.id=c95477af870f4dada1013a9a9b90bed1&scope.system=all&role.id=724184bfe3d1497590814ff09b602192 => generated 510 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:18.417053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.429781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.480598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb192-1aa7-48dc-8a6b-8b87eaf64dcf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.518411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.519233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.519515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.519793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.521071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.521780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.522558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.524280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 163/330] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:18.525666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fd32b-06af-45e7-bfb1-af911ae5bd11 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.530691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.567998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.569043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.569324 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.569600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.570357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:18.570672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.572058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.581819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 168/331] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:18.587488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.642288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c0bb4-977c-4c83-9a3c-f98c925ce5d2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.667603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:18.678898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.679608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.679889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.680162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] PATH_INFO: `/v3/groups` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.680834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.682402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.694361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaeb967-c78f-48e0-9397-d68b8d9e8323 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.697824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 164/332] 10.222.0.55 () {60 vars in 1134 bytes} [Tue Jun 28 11:51:18 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:18.729339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.730257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.730529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.730802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] PATH_INFO: `/v3/users/794ba44b48ce47f481df634e72172377` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.732433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] RBAC: Authorizing `identity:get_user(user_id=794ba44b48ce47f481df634e72172377)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.734414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e30c-8d14-4646-b383-232cf60af65d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.736258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 169/333] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:18 2022] GET /identity/v3/users/794ba44b48ce47f481df634e72172377 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:18.742164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:18.863072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f63546-707e-4d5d-a249-005a6c02645e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:18.897508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:18.898417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:18.898693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:18.898963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] PATH_INFO: `/v3/system/users/794ba44b48ce47f481df634e72172377/roles/66b89bc3a380455493a8eda82c8404a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:18.900512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=794ba44b48ce47f481df634e72172377, role_id=66b89bc3a380455493a8eda82c8404a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:18.901963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e4a75-9d4f-467f-b270-b45e7f07ff35 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:18.911660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 165/334] 10.222.0.55 () {58 vars in 1334 bytes} [Tue Jun 28 11:51:18 2022] PUT /identity/v3/system/users/794ba44b48ce47f481df634e72172377/roles/66b89bc3a380455493a8eda82c8404a3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:51:21.207219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7ffa-739d-4bc7-b896-4e2fe7971271 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.207714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7ffa-739d-4bc7-b896-4e2fe7971271 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.207714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7ffa-739d-4bc7-b896-4e2fe7971271 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.209366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 170/335] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:21.214855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.215645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.216089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.237071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:21.238574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:21.299334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:21.315986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ff11f-ba2b-4f55-b9dc-8ee28ae3004c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.316697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ff11f-ba2b-4f55-b9dc-8ee28ae3004c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.317315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ff11f-ba2b-4f55-b9dc-8ee28ae3004c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.319398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 171/336] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:21.325033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 166/337] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:21.326241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.326535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.326816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.353088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:21.354525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:21.363049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:21.397556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:21.478173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 172/338] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:21.510859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:21.532321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a586a-5e19-4d72-b8cb-071bfcd63753 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:21.577014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:21.578134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.578546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.578981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.605236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:21.618386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:21.619573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:21.620603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:21.623070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 167/339] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:21.628879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:21.678457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:21.756491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ce926-2062-41c4-a2d7-510d7bed9df7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:21.765206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ca385-bf68-42c1-b933-230944127620 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:21.811261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:21.812642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.814230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.814737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.815886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:21.817194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:21.819470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67ef61-639f-45ca-8434-7d544582d69e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:21.833919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 168/340] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:21.839087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2c139-eaae-49cf-8d10-507d22f77056 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.839909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2c139-eaae-49cf-8d10-507d22f77056 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.840878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2c139-eaae-49cf-8d10-507d22f77056 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.841723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:21.842869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.843347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.843813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.847646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 169/341] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:21.849320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2c139-eaae-49cf-8d10-507d22f77056 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:21.870990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-9185b738-86fa-4224-90c9-7ebca40da04c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:21.880703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:21.881661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:21.882475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 173/342] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:21.883624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19cf9-b5fc-440d-9a94-014be55a7279 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.883801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19cf9-b5fc-440d-9a94-014be55a7279 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.884054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19cf9-b5fc-440d-9a94-014be55a7279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:21.909299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4be19cf9-b5fc-440d-9a94-014be55a7279 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:21.911376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4be19cf9-b5fc-440d-9a94-014be55a7279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:21.930295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:21.991595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 174/343] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:21.998639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae433c1-44be-4da8-a672-3bdf2757d57b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:21.999260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae433c1-44be-4da8-a672-3bdf2757d57b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:21.999689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae433c1-44be-4da8-a672-3bdf2757d57b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.005352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 175/344] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:22.006763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae433c1-44be-4da8-a672-3bdf2757d57b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.012929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2c139-eaae-49cf-8d10-507d22f77056 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.012929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.056578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.057430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.057716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.058000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.146003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae433c1-44be-4da8-a672-3bdf2757d57b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.159622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:22.161515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:22.196675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.197804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.198289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.198753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.199696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:22.200254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:22.201956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c600bc3-9ec0-4fc6-8b9b-aa74f5d578fd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:22.213208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 176/345] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:22.215296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.215780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.216268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.239586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:22.241215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:22.252641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.290914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 170/346] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:22.290914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.292075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.326216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 177/347] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:22.327511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.401853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4a14f-fe3c-4765-8468-50d18b92dc4d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.439467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.440292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.440558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.440826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.451833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:22.459587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c43a87-3fcc-4b6e-884f-c8e53933d2f4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:22.460587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:22.461601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 28 11:51:22.462403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:22.463012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 171/348] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:22.464388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.500623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.501776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.502296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.502770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.516186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.538432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.576374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:22.578285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:22.583837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eca35-36da-4db5-8bc6-e38a89ef7f87 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.622171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.623049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.623360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.623686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.625874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:22.627361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:22.629619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 172/349] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:22.630262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.705333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 178/350] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:22.706542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.742831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cdd48-6406-4cb2-8a7c-96a13856eb35 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.755438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.787511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:22.795644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.796889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.797209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.797477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.814496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cfb4df0a-1e8d-4f33-8b6d-1ddb1b690e5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:22.822376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:22.823267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:22.824058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 173/351] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:22.825948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4948-e316-4366-af37-32c3531cc61f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.826225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4948-e316-4366-af37-32c3531cc61f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.826539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4948-e316-4366-af37-32c3531cc61f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.830170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 174/352] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:22.830170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b4948-e316-4366-af37-32c3531cc61f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.847418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbf175-5c05-4746-b500-8271fef3ee0c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:22.895204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:22.895900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:22.896202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:22.896447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:22.897881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:22.898204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:22.899800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:22.910923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 179/353] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:22.911758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:22.963284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b4948-e316-4366-af37-32c3531cc61f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.004987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.006234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.006736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.007020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.017408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.019503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.025127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 175/354] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.025127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.043356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598a570-27a0-4820-8de6-9d1f8fa8c488 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.046815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.086983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.087783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.088059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.088329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.089430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.090145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.091734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a08814-f728-4b30-b6ca-ea93b88d1d95 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.093686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 180/355] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:22 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.094872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.095154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.095421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.113540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:23.114840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:23.147887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2c29d-05d5-4b7e-b81e-83b97a6508e1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.190072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.191529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.191529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.191529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.192049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:23.192352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.194423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db979fc-402a-4321-beb2-be71b774f5a1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.194774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 181/356] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:23.197054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.202670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 176/357] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.203818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0c5f0-e242-4f7b-ade8-3a37a2dda2d5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.204098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0c5f0-e242-4f7b-ade8-3a37a2dda2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.204367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0c5f0-e242-4f7b-ade8-3a37a2dda2d5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.205965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 177/358] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:23.207109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0c5f0-e242-4f7b-ade8-3a37a2dda2d5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.330522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46109db8-7068-4d3c-a12c-bca38b9e333b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.331335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0c5f0-e242-4f7b-ade8-3a37a2dda2d5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.332096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.350103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.369847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.370511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.370793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.371065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.371904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=724184bfe3d1497590814ff09b602192, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:23.372216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.373717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.376419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.377336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.377617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.377889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] PATH_INFO: `/v3/users/794ba44b48ce47f481df634e72172377` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.379131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] RBAC: Authorizing `identity:get_user(user_id=794ba44b48ce47f481df634e72172377)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.379715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 178/359] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=724184bfe3d1497590814ff09b602192 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.380516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.386147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214c2b2-71ef-4746-9ae7-545375799a10 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.388198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 182/360] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/users/794ba44b48ce47f481df634e72172377 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.389562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.389861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.390140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.416281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:23.417701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:23.523175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828be6b-66c3-485e-b035-44dad07c3882 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.527118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 183/361] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:23.528972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.567839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.567839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.567839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.569111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.570583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.572243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 179/362] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:23.573091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.635494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.661690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.665211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332c6b7-9126-4c1a-9cb1-ecd18f2f5255 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.698556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaafc6c-524c-482d-a15b-d0ca3f21f874 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.713289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.714323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.714875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.715191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.744078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:23.745012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:23.745226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:23.745502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:23.765202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:23.774768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:23.775644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:23.776439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:23.777174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 180/363] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:23.777920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.807077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:23.809540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:23.937715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:23.943250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 184/364] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:23.944141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:23.954929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4b9be-82f0-4a13-b45a-0e00504024e6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:23.966490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.001446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.002222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.002575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.002966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.003879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] RBAC: Adding query filter params (role.id=66b89bc3a380455493a8eda82c8404a3, user.id=794ba44b48ce47f481df634e72172377, scope.system=all) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:24.004629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.005906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.022643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 181/365] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/role_assignments?user.id=794ba44b48ce47f481df634e72172377&scope.system=all&role.id=66b89bc3a380455493a8eda82c8404a3 => generated 510 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.024941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.077201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c4170-46b8-4668-b562-02c6faff523c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.117277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.118522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.119008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.119496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.132738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:24.141342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:24.142312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:24.143112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:24.146205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 185/366] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:24.147386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.153988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e17101-eebb-4248-8499-a1163a733212 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.200620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.200620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.200620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.200620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.201063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:24.201063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.201063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.210629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 182/367] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.211691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.234026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.255642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.304566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bea87-5ef9-4fd0-b443-0aa277bd0f15 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.338564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd649be-ea3f-457b-b574-13270fff2aa6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.347418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.348996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.349482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.349967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.350972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:24.351524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.353245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.364060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 186/368] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.368508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.380655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.381801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.381910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.382114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.383754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.385314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.392641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 183/369] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.393715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.489442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d773d-41ff-4540-bd47-de1d0133dff3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.505537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e5e5c-688c-4ed6-a1c5-fa5765d255aa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.508056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.522759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.543136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.543824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.544103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.544373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.545159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:24.545466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.546986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.548789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.549732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.550198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.550666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.555897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 187/370] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.556721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.633901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.635797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.694508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9f38b-49a9-4dda-9aec-a2d077f2e732 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.742902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.744371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.744870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.745560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.747368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.750285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.753414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 188/371] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.754958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.764400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.773065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 184/372] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:24.774193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:24.833355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:24.883850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8aeb-dbca-4580-94ad-0ac2a9cac576 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.901026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302061d4-4531-428c-97dc-f4b3e60575b7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:24.925519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.926186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.926469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.926743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.945071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:24.946249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:24.946673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:24.947253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:24.948613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=724184bfe3d1497590814ff09b602192, scope.project.id=c837898897aa468aa0e08029f92081cd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:24.949356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:24.951124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:24.958036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 185/373] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=724184bfe3d1497590814ff09b602192 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:24.959018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.039134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.040766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.104184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:25.121272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:25.169311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 189/374] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:25.171567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.228842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ddfd3-d5b5-4216-b5d3-31e8b764dd0d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.270346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.271239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.271522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.271795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.275985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.277737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.279808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 186/375] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:24 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.286751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.297906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2ca09-c8e6-4e81-af9f-40b79e385162 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.378159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.379044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.379321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.379600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] PATH_INFO: `/v3/projects/9c70d24580e841bb8af0388fc0999482` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.387036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] RBAC: Authorizing `identity:get_project(project_id=9c70d24580e841bb8af0388fc0999482)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.389759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.394576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 190/376] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:25 2022] GET /identity/v3/projects/9c70d24580e841bb8af0388fc0999482 => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.398743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.433321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:25.517343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8b6df-95a6-41fd-8d6b-02baa573ea1c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.554976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee36626-45f2-4eb7-84f1-d2eda19ce618 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.577757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.578775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.579061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.579338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] PATH_INFO: `/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.589966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] RBAC: Authorizing `identity:get_group(group_id=b9ff178d5b0b4142a6cbbcd96c396d6e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.591675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.597175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:25.603213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 191/377] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:25 2022] GET /identity/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.605100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.606656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.607135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.607591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.609847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.610292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.612042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.614460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 187/378] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:25 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.620952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:25.736350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9945d-bc3d-4e34-8204-f722e76a6239 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.749500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:25.756919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba2b47-d164-44b9-b4ea-6cd5530ce3df None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:25.775063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.775973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.776425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.776868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.777891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=c837898897aa468aa0e08029f92081cd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:25.778389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.779990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc594e3-9634-46bc-bfe3-35a2303f202a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.793021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 188/379] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:25 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.812836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:25.813540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.813827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.814101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.815016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] RBAC: Adding query filter params (group.id=b9ff178d5b0b4142a6cbbcd96c396d6e, role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=9c70d24580e841bb8af0388fc0999482) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:25.815331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:25.816893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ec642-7427-4030-b3b1-f11f07abdd22 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:25.829466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 192/380] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:25 2022] GET /identity/v3/role_assignments?group.id=b9ff178d5b0b4142a6cbbcd96c396d6e&scope.project.id=9c70d24580e841bb8af0388fc0999482&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:25.847587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84011f89-f49a-4c8d-8832-b282e9dc503c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.847914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84011f89-f49a-4c8d-8832-b282e9dc503c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.848137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84011f89-f49a-4c8d-8832-b282e9dc503c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.849856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 189/381] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:25.855576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:25.856288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:25.856568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:25.883936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:25.885233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:25.998628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 193/382] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:26.029639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84011f89-f49a-4c8d-8832-b282e9dc503c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:26.121912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:26.173837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84011f89-f49a-4c8d-8832-b282e9dc503c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:26.221644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:26.222479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.222765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.223038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.234720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:26.242462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:26.243322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:26.244559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 190/383] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:26.250166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:26.378621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:26.401147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726afb4-35aa-4a70-8500-2a31cffbeee9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:26.435785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:26.436558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.436841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.437145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.437906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:26.438216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:26.439603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:26.448538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 194/384] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:26.454066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:26.612019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95b37-4489-4fda-a693-8399ac2ca54f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:26.641319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:26.662608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:26.663498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.663775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.664045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.665264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:26.666728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643d49b-f22e-49f0-ac05-adb697a709cb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:26.668371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 191/385] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:26 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:26.674240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:26.741444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6864a-f70b-4605-be28-bf0a0a6dbb6c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.741756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6864a-f70b-4605-be28-bf0a0a6dbb6c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.741989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6864a-f70b-4605-be28-bf0a0a6dbb6c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.743699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 192/386] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:26.748920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.749318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.749707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.792222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461d4f0-683e-4a4e-bd8c-7d978999a957 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:26.794135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:26.795443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:26.826869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:26.827862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:26.828146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:26.828459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:26.829688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:26.831143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:26.833272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 195/387] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:26 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:26.838623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:26.865261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:26.966395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b175f-d6b3-4344-94f4-493f73b8d81c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:26.970085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 193/388] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:27.008283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.009197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.009473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.009740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/groups/21552575fed34138ab16f0384742d664/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.012823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:27.014174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, group_id=21552575fed34138ab16f0384742d664, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:27.015940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:27.040141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 196/389] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Jun 28 11:51:26 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/groups/21552575fed34138ab16f0384742d664/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:27.127228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc78c21-ce19-4eba-9a40-99bdbf987897 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:27.149987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:27.166001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.166848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.167131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.167416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.180405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:27.189175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:27.191370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:27.193363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 194/390] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:27.197006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:27.273309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:27.308449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9df96e-9893-4165-b0e5-a3c0dc356f18 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:27.347043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.347744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.348027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.348301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.349108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:27.349417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:27.350854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:27.361790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 197/391] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:27.367611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:27.473995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec6f39-5688-4b06-b584-9e7c7b40bd54 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:27.508957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.509846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.510129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.510400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.511644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:27.513192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:27.515288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 195/392] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:27 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:27.520808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:27.627311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ccff3-fc8d-4b72-b84f-0c5d912cb098 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:27.660843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:27.664572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.665729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.666244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.666580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.667959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:27.669475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df51df8-0b80-4fe2-aa38-09bb44be92c7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:27.671605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 198/393] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:27 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:27.677450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:27.729094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:27.756086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cc3aa-2980-40d1-b98e-126535849677 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.756428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cc3aa-2980-40d1-b98e-126535849677 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.756676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cc3aa-2980-40d1-b98e-126535849677 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.758355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 199/394] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:27.763363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.763643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.763916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.801826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:27.803036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:27.824326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f22f2d-4d23-4b7c-8c74-17b551025ccb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:27.886441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:27.887265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:27.887541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:27.887812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/groups/21552575fed34138ab16f0384742d664/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:27.892629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=450fcdaebc514486a6a72cfdb95e4e9f, group_id=21552575fed34138ab16f0384742d664, role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:27.894436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:27.916498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 196/395] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Jun 28 11:51:27 2022] PUT /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f/groups/21552575fed34138ab16f0384742d664/roles/a0087dfad30d4fd095fbec15fc10286d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:27.949761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 200/396] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:27.986129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.124927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b96e-d8cf-48be-8767-d7c3b95d3d17 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:28.139320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:28.169656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:28.170492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.170811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.171227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.185486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:28.193457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:28.194326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:28.195594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 197/397] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:28.200792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.347925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708671-c6dc-49ad-a043-25a1c2e86ff7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:28.383445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:28.396447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:28.397475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.397944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.398400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.399370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:28.399879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:28.401495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d19848-8d08-4ac7-b6bf-58951737b0b0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:28.416941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 201/398] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:28.422630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.469507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a24da-b4a3-4d81-a8ab-224a222c29f0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.470108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a24da-b4a3-4d81-a8ab-224a222c29f0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.470556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a24da-b4a3-4d81-a8ab-224a222c29f0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.472101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 202/399] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:28.478153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661c0a-fd4b-44a0-ba74-7d5370cb700d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.478660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661c0a-fd4b-44a0-ba74-7d5370cb700d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.479097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661c0a-fd4b-44a0-ba74-7d5370cb700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.504523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bf661c0a-fd4b-44a0-ba74-7d5370cb700d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:28.505760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bf661c0a-fd4b-44a0-ba74-7d5370cb700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:28.591681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d33ff-f438-4dce-8730-75f62614efe4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:28.624310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 203/400] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:28.626712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5f-7d7b-4a6f-88ea-5b5cc18fb97a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.627185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5f-7d7b-4a6f-88ea-5b5cc18fb97a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.627671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de5f-7d7b-4a6f-88ea-5b5cc18fb97a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.631410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 204/401] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:28.642423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.643005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.643487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.648709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:28.650092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.651302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.652497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.655405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:28.657144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:28.659292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 198/402] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:28 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:28.660941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.671861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:28.672135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:28.725317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:28.826053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25794296-3818-4102-b4f0-cc7c898a3212 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:28.852925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 205/403] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:28.852925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.887180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:28.888117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:28.888657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:28.889789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:28.910045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:28.920872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:28.921927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:28.922714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:28.926268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 199/404] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:28.939501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:28.952884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.013308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d052d6-e233-4bfd-8b5d-f2127a1c27f5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.064717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.065632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.065912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.066205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.068066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.069563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.076285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 206/405] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:28 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:29.076285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.142118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.149447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097f5a6-eaa1-41a0-9211-cf61d3667d67 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.187655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.188878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.189456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.189917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.201593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:29.209756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda3ce9-e751-4ecf-951b-00f1259cfad3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:29.211007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:29.211868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:29.212650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:29.215544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 200/406] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:29.216529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.228814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.247625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.248304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.248586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.249003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.249950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:29.250260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.251666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.265404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 207/407] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:29.265995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.336196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ec3fa-aaa3-4309-bf0a-6c83816ed4c7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.373077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71648ba-1343-4554-aa41-052c2f36fc1f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.375408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.377413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.377883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.378319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/groups/21552575fed34138ab16f0384742d664/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.380454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, group_id=21552575fed34138ab16f0384742d664, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.382458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.401525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 201/408] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Jun 28 11:51:29 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/groups/21552575fed34138ab16f0384742d664/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:29.402888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.414727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.415622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.415977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.416838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.417355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:29.417739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.419371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.434685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 208/409] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:29.437507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.438720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.505799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.577272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446549-c1e9-470c-a3aa-611af06048c0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.590200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531a15b-b544-4461-862d-64fcec39c4fa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.622483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.625003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.626063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.626530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.628724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.630532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6d2c0-fc1e-41d8-8ddf-7b661f31608d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.632398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 209/410] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:29.636955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.636955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.636955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.636955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] PATH_INFO: `/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.637297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] RBAC: Authorizing `identity:get_group(group_id=b9ff178d5b0b4142a6cbbcd96c396d6e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.637902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.638907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4112e5-9488-46e6-9f56-d62c5dcd79d1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.639400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4112e5-9488-46e6-9f56-d62c5dcd79d1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.639933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 202/411] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:29.640336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4112e5-9488-46e6-9f56-d62c5dcd79d1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.642160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 210/412] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:29.644677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4112e5-9488-46e6-9f56-d62c5dcd79d1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.644803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.738771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.766090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed142c0-3e5f-4264-b768-3f9e4e24fb35 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.778474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4112e5-9488-46e6-9f56-d62c5dcd79d1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:29.791181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:29.817085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.818337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.818819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.819287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.820309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:29.820947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.821245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.821526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.821801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] PATH_INFO: `/v3/projects/9c70d24580e841bb8af0388fc0999482` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.822995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] RBAC: Authorizing `identity:get_project(project_id=9c70d24580e841bb8af0388fc0999482)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:29.823380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.824493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36a099-f6d5-46f5-a335-86fefd518e3b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:29.830583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 211/414] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/projects/9c70d24580e841bb8af0388fc0999482 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:29.830583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:29.830583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:29.830583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:29.831259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 203/414] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:29 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:29.832293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.846447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:29.847715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:29.928987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 212/415] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:29.929922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:29.961671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dff45-35c0-4b69-8488-ba1014fc50af None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.001216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.002650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.003119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.003579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] PATH_INFO: `/v3/projects/9c70d24580e841bb8af0388fc0999482/groups/b9ff178d5b0b4142a6cbbcd96c396d6e/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.006204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c70d24580e841bb8af0388fc0999482, group_id=b9ff178d5b0b4142a6cbbcd96c396d6e, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:30.008024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:30.026623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 204/416] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Jun 28 11:51:29 2022] PUT /identity/v3/projects/9c70d24580e841bb8af0388fc0999482/groups/b9ff178d5b0b4142a6cbbcd96c396d6e/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:30.028535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:30.052450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.052652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d72e08-81ea-4a36-b7a3-10e47443948a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.066338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.098413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.099378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.099663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.099939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd/groups/21552575fed34138ab16f0384742d664/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.102398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] RBAC: Authorizing `identity:create_grant(project_id=c837898897aa468aa0e08029f92081cd, group_id=21552575fed34138ab16f0384742d664, role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:30.103886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:30.128932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 213/417] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Jun 28 11:51:29 2022] PUT /identity/v3/projects/c837898897aa468aa0e08029f92081cd/groups/21552575fed34138ab16f0384742d664/roles/a0087dfad30d4fd095fbec15fc10286d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:30.207927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559b368-8dfc-46b5-9cb4-5739e6220465 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.260234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.261196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.261473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.261739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.280491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3134ded3-04f6-40c5-9c2c-06dbda5a2605 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:30.288642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:30.289618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:30.290871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 205/418] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:30.298801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:30.362422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0700392-ed6e-4ee9-807f-daa73a5f7ec1 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.362745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0700392-ed6e-4ee9-807f-daa73a5f7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.362973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0700392-ed6e-4ee9-807f-daa73a5f7ec1 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.364639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 206/419] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:30.369354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.369636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.369907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.400690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:30.401909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:30.435500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3084f8-86a6-4156-a278-0982faf61ddc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.476740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.481069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.481069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.481069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.481069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:30.481069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:30.481431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:30.491937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 214/420] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:30.502061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:30.518398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 207/421] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:30.538280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.554077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:30.581287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.667999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d29b3-1934-4bb6-a415-fb1405baefed None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.686949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aed0d5-558e-4bc0-a94d-b3089152ea1a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:30.723752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.724812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.725252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.725687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.730549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:30.731579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:30.731873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:30.732136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:30.738463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:30.746954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:30.748481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:30.749578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:30.752015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 208/422] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:30.757768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:30.873581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.881982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:30.883466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:30.936410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:30.941743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800387c-a845-4a55-abf3-02d066b50b05 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.009712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.010391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.010669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.010936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.011692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:31.012055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.013536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.027572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 209/423] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:31.031077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.062086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 215/424] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:31.067168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.179370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fc8ae-2b86-4a5f-ac96-365aaf00ee8e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.237878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:31.250654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.251376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.251657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.251929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.264525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421e2af-fde6-4090-bb06-927bf8dd8a24 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.296147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:31.305893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.306818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.307100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.307377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.308546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.312925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.312925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 216/425] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:31.317847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.427114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.428615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.457932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc526-5dca-45a4-84b5-c9a4bcd2b40f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.496330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.497226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.497503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.497881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.499152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.500773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.502806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 217/426] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:31.514153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.587276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:31.625802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0e3ed-2aad-4179-b63b-fb06440b37bf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.651989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 210/427] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:31.657962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.673140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.673835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.674082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.674350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.675182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=724184bfe3d1497590814ff09b602192, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:31.675491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.677122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ef406-b657-4f65-a5ef-c7bc0a482d1e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.689129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 218/428] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=724184bfe3d1497590814ff09b602192 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:31.700328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:31.806021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4483051-9804-4f23-8274-289ba58c9266 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.806364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4483051-9804-4f23-8274-289ba58c9266 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.806567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4483051-9804-4f23-8274-289ba58c9266 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.807950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 219/429] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:31.822703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.823000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.823254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.829476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d272-2889-4eb9-a000-0dbdaf1e4cfa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:31.856980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:31.859719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:31.888059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:31.889200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:31.889310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:31.889543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] PATH_INFO: `/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:31.890975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] RBAC: Authorizing `identity:get_project(project_id=450fcdaebc514486a6a72cfdb95e4e9f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:31.892327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:31.894372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 211/430] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/projects/450fcdaebc514486a6a72cfdb95e4e9f => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:31.904927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:31.956401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 220/431] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:31.981360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.018380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79712c-2293-43de-8fa1-cf8ada06ddc0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.021792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.056875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.058350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.058805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.059240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.061326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:32.063863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:32.065815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 212/432] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:32.066668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.072518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.126325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164618d7-f317-46d5-a712-ea4ee2e7ef34 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.176209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.177068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.177354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.177628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.189876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:32.199579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:32.200506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:32.205035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 221/433] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:32.210904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.218845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb3553-33ec-4f32-8282-4aa5013e042f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.259455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.260181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.260465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.260785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.261664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:32.261980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:32.263433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:32.276932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 213/434] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=450fcdaebc514486a6a72cfdb95e4e9f&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:32.337102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62002b7b-8473-4a46-8f98-2244e26d2b09 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.359676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.377249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.377972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.378514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.378796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.379565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:32.379878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:32.381326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13ad4e-511d-4f6a-81fd-a8e6ebe9ba7d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:32.393355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 222/435] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:32.410546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.446726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.453925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764c0c4-a3b5-4336-b55f-5fe18f183c1f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.454268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764c0c4-a3b5-4336-b55f-5fe18f183c1f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.454490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764c0c4-a3b5-4336-b55f-5fe18f183c1f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.456080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 223/436] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:32.461936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.462235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.462490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.485481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:32.487692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:32.538741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea2a7-dd71-478f-a0e4-4dc4ab0e8c74 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.584468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 224/437] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:32.584680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.584680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.584919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.585216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.607142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.666401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:32.668208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754385e0-6338-4bd6-8edc-932a77d2fac5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:32.696541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.727899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15f6c9-81bf-48ad-833e-8709f3d9808f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.747578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.777641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.778762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.779050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.779323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.791340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2aac74b5-1979-48b4-9cde-a77c4ef75074 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:32.799629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:32.800523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:32.801406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 225/438] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:32.805429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bec5b2-9af0-4f66-a601-11dab774233e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.805429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bec5b2-9af0-4f66-a601-11dab774233e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.805429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bec5b2-9af0-4f66-a601-11dab774233e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.808657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 226/439] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:32.809513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bec5b2-9af0-4f66-a601-11dab774233e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.812237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 214/440] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:32.814429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.814727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.815001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.833016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:32.834165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:32.900264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 215/441] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:32.901266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.915579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bec5b2-9af0-4f66-a601-11dab774233e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:32.949248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:32.949839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:32.950115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:32.950386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:32.951125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:32.951428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:32.953147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:32.962212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 227/442] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:32.962333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:32.962830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:32.997476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.011575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb4c90-e5ab-4b69-b001-9ff37d82b68b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.049967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.050838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.051117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.051389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.052591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.054099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.055821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 216/443] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:33.056577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.090367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973bd7-96d4-40e0-b965-58d03d9e4e01 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.129386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.130215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.130493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.130761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.140478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:33.148670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:33.149727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 228/444] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:33.150702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.174516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a53cdf-6c8c-4e49-acc8-eeb999deb9f9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.197282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.209053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.209642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.209921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.210189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.247817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.255227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72cee1-5efd-4097-87f3-9c98177b742b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.276698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.278210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.291459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.292345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.292632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.292931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.294147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.295630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.297301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 229/445] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:33.298062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.374485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 217/446] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:33.375526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.405443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.408827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5ec1-0b15-4c75-ae85-bca64c0d88d3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.453898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.454604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.454871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.455144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.455875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:33.456181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.457598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.465476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 230/447] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:33.466194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.491827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.502455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c28-30e8-409d-8231-07deb4bc5a9f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.536400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.537112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.537417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.537675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.538503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=724184bfe3d1497590814ff09b602192, scope.project.id=c837898897aa468aa0e08029f92081cd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:33.538814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.540274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.546696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 218/448] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=724184bfe3d1497590814ff09b602192 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:33.547476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.590477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518758-b651-49e3-afbd-2e7e2063d0d0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.633965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.634883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.635161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.635429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] PATH_INFO: `/v3/projects/9c70d24580e841bb8af0388fc0999482` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.636764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] RBAC: Authorizing `identity:get_project(project_id=9c70d24580e841bb8af0388fc0999482)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.638286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.641322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 231/449] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/projects/9c70d24580e841bb8af0388fc0999482 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:33.647333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.668876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.688365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579a52-98ce-4216-9486-cc15af113055 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.737757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.738345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.738620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.738997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.770146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.786005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ae72-c0df-4ccd-b946-b015f9c78082 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:33.828076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:33.829181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:33.829385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:33.829661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] PATH_INFO: `/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:33.831045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] RBAC: Authorizing `identity:get_group(group_id=b9ff178d5b0b4142a6cbbcd96c396d6e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.832738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.839426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 232/450] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/groups/b9ff178d5b0b4142a6cbbcd96c396d6e => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:33.841003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.844424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:33.846274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:33.948778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:33.965367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 219/451] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:33.971437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:33.975020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368d4b5-90af-4423-bdd0-8b6e0ea609e7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:34.012498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:34.014538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:34.014538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:34.014538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:34.015165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] RBAC: Adding query filter params (group.id=b9ff178d5b0b4142a6cbbcd96c396d6e, role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=9c70d24580e841bb8af0388fc0999482) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:34.015521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:34.017016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:34.025171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 233/452] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/role_assignments?group.id=b9ff178d5b0b4142a6cbbcd96c396d6e&scope.project.id=9c70d24580e841bb8af0388fc0999482&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:34.080657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41797b3d-c190-48e6-893e-d2ef58bc7d11 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:34.122112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:34.122983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:34.123256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:34.123519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] PATH_INFO: `/v3/projects/c837898897aa468aa0e08029f92081cd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:34.125135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] RBAC: Authorizing `identity:get_project(project_id=c837898897aa468aa0e08029f92081cd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:34.126594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:34.128359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 220/453] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:51:33 2022] GET /identity/v3/projects/c837898897aa468aa0e08029f92081cd => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:34.134016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:34.167100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:34.259663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc39d0-7313-4930-ad03-d74eaa1eacfc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:34.297241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:34.298586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:34.299055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:34.299502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] PATH_INFO: `/v3/groups/21552575fed34138ab16f0384742d664` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:34.301402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] RBAC: Authorizing `identity:get_group(group_id=21552575fed34138ab16f0384742d664)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:34.303042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad380c-52e0-4383-915b-d75cc6fe63ce None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:34.304692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 234/454] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Jun 28 11:51:34 2022] GET /identity/v3/groups/21552575fed34138ab16f0384742d664 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:34.314357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:34.389918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:34.449928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1faa9-eaf7-4f5e-bf54-7c084f1a33ad None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:34.495017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:34.495763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:34.496046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:34.496317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:34.497203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] RBAC: Adding query filter params (group.id=21552575fed34138ab16f0384742d664, role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=c837898897aa468aa0e08029f92081cd) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:34.497515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:34.499038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e9f399-4d59-4f76-8cfd-56ab032789e8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:34.505595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 221/455] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Jun 28 11:51:34 2022] GET /identity/v3/role_assignments?group.id=21552575fed34138ab16f0384742d664&scope.project.id=c837898897aa468aa0e08029f92081cd&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:37.300241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4691-28b3-48e6-9911-5be2e159f999 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.301168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4691-28b3-48e6-9911-5be2e159f999 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.301601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed4691-28b3-48e6-9911-5be2e159f999 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.303364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 235/456] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:37.309037 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f9b7-c166-4e70-b07a-16aa14225953 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.309778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f9b7-c166-4e70-b07a-16aa14225953 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.310187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f9b7-c166-4e70-b07a-16aa14225953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.337918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8fc2f9b7-c166-4e70-b07a-16aa14225953 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:37.339373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8fc2f9b7-c166-4e70-b07a-16aa14225953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:37.439640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 222/457] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:37.443769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ff92-4c16-4eef-9b27-efb85e67c486 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.444229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ff92-4c16-4eef-9b27-efb85e67c486 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.444642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ff92-4c16-4eef-9b27-efb85e67c486 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.446184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 236/458] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:37.451326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.451894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.452473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.473217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d395-e911-47d6-9155-e5d41d0f5f6b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.473816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d395-e911-47d6-9155-e5d41d0f5f6b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.474313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077d395-e911-47d6-9155-e5d41d0f5f6b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.476112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 237/459] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:37.477388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:37.478772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:37.481214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.481746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.482212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.502762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:37.504282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:37.533433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:37.557673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 223/460] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:37.558616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:37.576945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 238/461] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:37.594544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:37.624049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:37.694272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ecb56-40a2-43db-b1ef-967f5ce8c3d1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:37.709326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150deaa-7717-45c2-bda1-95ac8667c5b6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:37.743186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:37.743982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.744263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.744534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.753946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:37.755285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:37.755491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.755952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.756430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.756778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:37.774819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:37.776531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:37.784040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:37.785623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:37.786498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:37.786498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:37.786498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.786498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:37.795310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 224/462] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:37.796580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:37.802291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:37.803130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:37.803986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:37.803986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:37.803986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.803986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:37.808459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 239/463] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:37.809298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:37.835917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:37.915393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:37.915593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217f68d-6f33-486b-83b0-6fb32912917f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:37.944195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceeaca-1725-48b5-b2e4-c62a43c382ab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:37.961346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:37.962154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.962437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.962715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.975020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:37.976819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:37.980532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:37.981596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:37.981877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:37.982147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:37.982874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:37.983241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:37.984616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f317ef0-f599-4945-9cac-544781e2d161 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:37.995485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:37.996342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:37.997592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:37.997592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:37.997592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:37.997592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.002065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 225/464] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:38.002177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 240/465] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:38.002578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.003323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c9cb-5678-449e-ba07-bfa6e76219c4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.003611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c9cb-5678-449e-ba07-bfa6e76219c4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.003890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649c9cb-5678-449e-ba07-bfa6e76219c4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.009607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 241/466] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:38.010365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649c9cb-5678-449e-ba07-bfa6e76219c4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.098507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.122017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e2dc3-36f9-4541-bcdb-a3051695ad98 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.131318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649c9cb-5678-449e-ba07-bfa6e76219c4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.161513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.162317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.162599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.162867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.163607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:38.163911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.165380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.173122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.173796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.174080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.174758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.175536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:38.175772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.177176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99a0e1-1554-4f1c-8723-24b3af812f06 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.177394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 226/467] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:38.178247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.183470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.202569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 242/468] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:38.202569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.202569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.202569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.216389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:38.217841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:38.289497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 243/469] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:38.290331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.303779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78407479-82c5-4f70-aab5-cf098169de85 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.337224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.337975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.338267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.338533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.339241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.340713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.356779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.367945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 227/470] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:38.369218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.414827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055c02-9173-4308-a254-6d8f5c630c76 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.451670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.468580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.469655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.470081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.470490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.471349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.473034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646474b5-427d-467e-a332-4753c95eb8da None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.497017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823f6e3-0671-43ac-a662-cb4b0409dede None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.508999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 244/471] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:38.508999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3869-3ba0-4e2a-a405-b40f701b289e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.508999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3869-3ba0-4e2a-a405-b40f701b289e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.508999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3869-3ba0-4e2a-a405-b40f701b289e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.508999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 245/472] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:38.509874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3869-3ba0-4e2a-a405-b40f701b289e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.552609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.553602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.553882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.554307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.555094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.557308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2d7aa-97cf-4533-8029-2b7daff1e468 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.586688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 228/473] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:38.587994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.587994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.588185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.621059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:38.622702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:38.668686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3869-3ba0-4e2a-a405-b40f701b289e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.709110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.721238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.722092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.722354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.722624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.733093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 229/474] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:38.736016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.737543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.758197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:38.760991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:38.761856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:38.762728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:38.762728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:38.762728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:38.762728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:38.762728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 246/475] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:38.773487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:38.824547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:38.890561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9fa2-2983-4191-a683-8b15dcae8463 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.913834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641aa9b-48cf-41ce-a3dc-79a29dd12800 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:38.938246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.939174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.939600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.940036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.940965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.942518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.954813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.sql.core [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 28 11:51:38.954813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 28 11:51:38.954813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [parameters: {'id': '5ddb3c20588441328c0a5f2a629adcf3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 28 11:51:38.954813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80099) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 28 11:51:38.981578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:38.982529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:38.983065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:38.983536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:38.984417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:38.984867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:38.986415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jun 28 11:51:38.987687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return ref.to_dict() Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:51:38.988736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application yield resource Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.rollback() Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.force_reraise() Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise self.value Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application session.commit() Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 28 11:51:38.989879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._prepare_impl() Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.session.flush() Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._flush(objects) Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application compat.raise_( Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application flush_context.execute() Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rec.execute(self) Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 28 11:51:38.990798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:51:38.991711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application result.read() Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:51:38.992700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [parameters: {'id': '5ddb3c20588441328c0a5f2a629adcf3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:38.993706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 28 11:51:38.994559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:38.999712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 230/476] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Jun 28 11:51:38 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:39.010180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:39.024317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 247/477] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:39.029870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:39.086991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:39.174642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2be0af-78cd-4337-bd64-02a015b6d123 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:39.176248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:39.187290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd56be-0b80-4040-987d-cc8568146d91 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:39.229461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:39.230156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:39.230445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:39.231084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:39.231821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:39.233427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:39.265230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 248/478] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:39 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:51:39.278440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:39.279298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:39.279578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:39.279955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:39.311069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-71df3391-536b-4acb-a185-717e00a82011 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:39.319961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:39.320953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 28 11:51:39.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:39.322363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 231/479] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:51:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:39.329010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:39.463300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755bd12-dcf4-4d2f-b392-946518d84bc9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:39.501584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:39.502338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:39.502621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:39.502946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:39.503802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:39.504111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:39.506083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914242c-e14e-4a2f-a618-eceb238893fc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:39.515891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 249/480] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:40.906228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bdfd-c095-4b7a-aa9c-91e6756b563b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:40.907078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bdfd-c095-4b7a-aa9c-91e6756b563b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:40.907511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bdfd-c095-4b7a-aa9c-91e6756b563b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:40.909134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 232/481] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:40.914580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f41d7-bf22-41ec-926b-b389e374f7eb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:40.915009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f41d7-bf22-41ec-926b-b389e374f7eb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:40.915135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f41d7-bf22-41ec-926b-b389e374f7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:40.938707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-af2f41d7-bf22-41ec-926b-b389e374f7eb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:40.939916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-af2f41d7-bf22-41ec-926b-b389e374f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:40.945987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.029497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 250/482] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:41.065595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92bdfd-c095-4b7a-aa9c-91e6756b563b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.189626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92bdfd-c095-4b7a-aa9c-91e6756b563b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.199214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.237818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ac73-64b2-47f3-9a84-f796086fa88f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.238105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ac73-64b2-47f3-9a84-f796086fa88f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.238386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ac73-64b2-47f3-9a84-f796086fa88f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.239714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.240520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.240795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.241086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.245705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 251/483] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:41.254381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:41.266365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857386-cdfb-4910-9683-9bd460ee3066 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.266918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857386-cdfb-4910-9683-9bd460ee3066 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.267377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857386-cdfb-4910-9683-9bd460ee3066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:41.270524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:41.271392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.272190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:41.274787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 233/484] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:41.280206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.295265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-49857386-cdfb-4910-9683-9bd460ee3066 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:41.297041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-49857386-cdfb-4910-9683-9bd460ee3066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:41.370953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 252/485] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:41.372323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25838014-d123-4160-b4ea-d11c48ea0b83 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.372607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25838014-d123-4160-b4ea-d11c48ea0b83 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.372878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25838014-d123-4160-b4ea-d11c48ea0b83 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.382462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 253/486] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:41.390513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.390797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.391077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.401933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47dc41-5053-43ec-9a29-79d01f6fa25b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.418385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:41.419686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:41.446278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.447432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.448083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.448499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.449641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:41.449953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:41.451355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:41.462421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 234/487] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:41.463178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.495141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.502800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.513765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 254/488] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:41.521157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.606174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe760f83-ad36-4004-9f60-9ca47a68adad None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.635774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9f002-7993-45dc-b409-dde201f997f2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.652641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.654224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.655190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.655686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.672195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:41.674931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.675995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.676466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.676971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:41.680437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:41.681530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.682322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:41.686717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 235/489] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:41.687718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.693801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:41.695544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:41.720321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:41.721270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:41.722102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:41.722102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:41.722102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:41.722102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:41.722308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 255/490] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:41.723091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.796779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.798849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:41.812773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c1790-8c85-4a49-8791-fe2251dd0911 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.855398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2fc6c-9be9-458d-8343-5797cb67db58 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:41.856575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.857947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.859430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.859885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.873496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:41.884187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:41.885142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:41.885945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:41.886548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 236/491] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:41.892386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:41.900880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:41.901648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:41.901929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:41.902255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:41.903019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:41.903328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:41.905067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:41.913982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 256/492] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:41.914703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.022970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a98bd8-e25d-46ab-bac3-ed5f18b19f20 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.027401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7452145-917e-4323-93ef-55d1a151be50 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.042986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.061151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.061854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.062069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.062367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.063080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:42.063389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.064762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.071359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.072037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.072316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.072588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.073375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:42.073688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.075046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf29b11-676a-4283-ab04-1c1c98617114 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.079926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 237/493] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:42.081650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 257/494] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d53d-ed01-4064-a238-4db1c7b5f7d5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d53d-ed01-4064-a238-4db1c7b5f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d53d-ed01-4064-a238-4db1c7b5f7d5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 258/495] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:42.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb893ed-a1c4-498a-acee-5557f0fc36b2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.088634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb893ed-a1c4-498a-acee-5557f0fc36b2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.089059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb893ed-a1c4-498a-acee-5557f0fc36b2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.090565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 259/496] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:42.091102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb893ed-a1c4-498a-acee-5557f0fc36b2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.100043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.189318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982c74-341a-4b98-afc2-2ae36ded25fe None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.193317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb893ed-a1c4-498a-acee-5557f0fc36b2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.223995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.224793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.225086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.225361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.226581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.227377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.227657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.227938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.235734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.236181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.237215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.237681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:42.252676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.253646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:42.254557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 260/497] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.255776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ad9d7d1f-7f8f-4287-ab81-5bb4e1af8889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:42.255847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.256598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.259605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:42.259605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:42.259605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.259605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:42.260610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 238/498] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.261913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e520-a577-47e1-b50b-fe0fe193aa65 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.262195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e520-a577-47e1-b50b-fe0fe193aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.262474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e520-a577-47e1-b50b-fe0fe193aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.278795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-98d9e520-a577-47e1-b50b-fe0fe193aa65 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:42.280108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-98d9e520-a577-47e1-b50b-fe0fe193aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:42.293062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.338355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.351563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 239/499] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:42.353079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.353356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.353625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.368535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd587de2-d620-4921-88f1-5a2923746d37 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.371765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:42.373065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:42.404214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.405201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.405471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.405824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.416355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.417846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:42.434641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.435567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 261/500] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.437604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.453925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 240/501] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:42.454731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.527315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.540384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f65ed-41c1-4d7b-8b7b-da31de873b45 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.558210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ebe26-1d41-4486-84a5-1a32c42af444 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.573504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.574175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.574453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.574724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.575569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:42.575877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.577504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.582270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.594755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.595427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.595712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.595986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.596718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:42.597080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.598482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.610168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 241/502] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:42.610964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.615667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 262/503] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:42.616497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.715608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02153d4-495a-408f-80ad-77e8049c6fda None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.723425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b076e25-d8e2-4ae4-80b7-fecf51f07d5a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.749947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.750753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.751042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.751308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.758587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.759338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.759690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.759972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.760700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:42.761036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.761900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.762464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.763358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.774416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 263/504] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:42.775255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:42.781341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.782181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.783022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:42.783022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:42.783022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:42.783022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:42.783022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 242/505] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.783687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.784519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.823471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:42.878452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd23ccf-06a1-4fe2-beb9-f05c532e6193 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.887911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19cfc-379b-4cb2-a848-d950c9218d7f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:42.911321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.912121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.912407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.912682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.921418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:42.922221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:42.922500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:42.922770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:42.922917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:42.930504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:42.930693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.931665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:42.932482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 264/506] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.933171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:42.933247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:42.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:42.949773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:42.950663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 243/507] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:42.951207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.004353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.038761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fbcbb-17dc-47e1-82ee-6bb73c461a8e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.044722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.052055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86095a-339a-40a5-9425-2c4444fb5a1a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.072447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.073160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.073444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.073724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.085282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.086081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.086352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.086624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.094128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.095589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:43.110396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:43.111289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:43.112170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 244/508] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:43.112729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.141187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.142738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.205838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.213840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df777f-5769-4e49-9562-96c4ab935f59 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.242875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 265/509] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:43.243703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.247760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.248408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.248740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.249051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.249779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:43.250088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.251479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.263735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 245/510] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:43.264427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.280939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.348460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076187c1-b4fb-4197-8357-c807657c5b8b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.364607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1fb-7644-4858-b6e1-60ca0b070e1e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.382343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.383011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.383292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.383563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.384328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:43.384629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.386062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.394148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 266/511] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:43.394844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.402220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.403137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.403446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.403744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.404608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:43.405023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.406474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.424281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.443418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 246/512] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:43.444261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.497880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c758-f24d-457b-be06-8ce4de445166 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.517333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.530403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.531066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.531350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.531620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.532460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:43.532763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.534237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.547145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78ce6-84cf-4b4f-b711-dab0284fe6b3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.550803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 267/513] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:43.551572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.581200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.581996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.582271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.582557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.592616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.594091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:43.611778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:43.612622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:43.613526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:43.613526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:43.613526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:43.613526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:43.613526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 247/514] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:43.613994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.639799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.656516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed47c6-5e3f-495b-a655-8d84cb5ad412 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.691865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.692546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.692825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.693149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.693829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.695284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.714539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 268/515] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:51:43 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:51:43.715344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.715776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029dd37-c7a6-40e3-8ebe-c54ebfa4a6d1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.756083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.772187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.772751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.773067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.773414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.790878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.792423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:43.812514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:43.813706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:43.814547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:43.814547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:43.814547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:43.814547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:43.814547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 248/516] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:43.816061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:43.846110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b4281-5b0a-4174-b3e8-482fa5a93e7f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.888798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.889562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.890402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.890402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.904700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:43.932835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0d53-3a62-4af3-a47a-3ebe9f66d610 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:43.966785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:43.967549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:43.967897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:43.968177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:43.968507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:43.970009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.016875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.040861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.042289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.075514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 269/517] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:44.076921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.121953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.151344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 249/518] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:44.153844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.180381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb48d0-168e-49e4-9481-9eecf9bf6b7f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.220368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.221078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.221364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.221640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.222369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:44.222674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.224079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.236412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 270/519] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:44.236925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.237277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.266759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620711-a0f9-404d-bfc6-430efc125949 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.304081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.304695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.305085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.305358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.306078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:44.306381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.307884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.324348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 250/520] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:44.325235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.338485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf15ff-779c-487c-a658-c4751cc5dd58 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.340029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.373805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.374770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.375046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.375322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.385947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.387416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:44.406288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:44.407187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:44.408056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:44.408056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:44.408056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:44.408056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:44.408056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 271/521] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:44.408317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.438264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb4d2-7506-42d9-82ce-5b0a284a4ce5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.476600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.477446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.477728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.477998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.481124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.491271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.492740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:44.510418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:44.511477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:44.512506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:44.512506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:44.512506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:44.512506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:44.515527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b521aa-6de7-46fb-867a-86f245a773b7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.516315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 251/522] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:44.517173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.552662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.553497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.553787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.554044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.560817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.562317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:44.577461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:44.578515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:44.579565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:44.583369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 272/523] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:44.584139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.600041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.625726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21572bc-c0f6-431c-b3fa-7a1a754349fd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.659852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.660662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.660970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.661246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.671808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.673515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:44.688415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:44.689365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:44.690280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 252/524] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:44.690871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.698198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef850eaa-1840-41fd-8b8a-0299cb227ff4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.737224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.738070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.738449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.738822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.739643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:44.740045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.741552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.742182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.758500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 273/525] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:44.758500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.811189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091b708-47c8-4ce6-b209-e7469eded929 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.856074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.856767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.857088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.857364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.858169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:44.858478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.859892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.860349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:44.873561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 253/526] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:44.874226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.892188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a28e0d-b3df-42f2-963e-8681969b2d51 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.925896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:44.926667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:44.927017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:44.927293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:44.928080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:44.928383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:44.929871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:44.952695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 274/527] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:44.952695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:44.979053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d167d-1937-44ca-949a-a74bfdfa3dc5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:44.989731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.017961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.018647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.018928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.019201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.020043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.020356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.021849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.042184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 254/528] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.043492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.060539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c475-3e17-47eb-b4e2-32f6c369a571 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.097430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.098163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.098438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.098725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.100957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.107812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.110232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:45.135895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:45.137024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:45.137988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 275/529] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:45.138780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.160226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80479d6-b7c8-4248-a53e-2b4e81340945 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.196458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.197459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.197791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.198151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.206760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.208245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:45.223563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:45.224591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:45.225611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:45.229364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 255/530] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:45.231104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.249784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70019210-4c04-4a92-8d93-c12c0dbaabc4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.261047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.285065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.285728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.286006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.286276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.287117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.287428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.288873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.294429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 276/531] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.295197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.354171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38551b0f-3c0b-492e-b6f2-26949d6d1e3b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.381025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.404598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.406242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.407325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.407820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.433317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89e21d-950f-401b-a87f-2fabbb0850c6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.474718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.475396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.475681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.475954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.476749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.477117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.478578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.497706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 277/532] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.498813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.509352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.512012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.544540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.614912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229e16-103e-4dde-a5e8-c65729dde489 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.631706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 256/533] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:45.632636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.634781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.654412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.656230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.656934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.657417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.658428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.658934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.660602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.677004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 278/534] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.678206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.742576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a2bc3-0954-42f7-80e3-85f1c205a72a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.772996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.788346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0da2b3-a735-4f96-983f-de35606a649d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.789770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.790442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.790718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.790989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.791833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=0f5e14d9d362461dae34b8330718002b) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.792145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.793611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.801230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 257/535] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/role_assignments?user.id=0f5e14d9d362461dae34b8330718002b&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.801230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.829658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.833737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.834275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.834686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.848370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.850180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-94262d4d-a625-47f6-9434-0d533ccbee92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:45.871232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:45.872114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:45.872971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:45.872971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:45.872971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:45.872971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:45.876190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 279/536] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:45.877970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1684b-bef0-4ff3-bb35-0b6210a570bc None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.878415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1684b-bef0-4ff3-bb35-0b6210a570bc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.878839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1684b-bef0-4ff3-bb35-0b6210a570bc None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.880728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 280/537] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:45.881758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1684b-bef0-4ff3-bb35-0b6210a570bc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:45.917446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:45.931977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebf35f-6ee3-4d4f-b20c-247f77bc6854 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:45.970173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:45.970956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.971307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.971586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:45.972428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=e6c561ff3ab84f99b81eaccc9d9f2cad) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:45.972742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:45.975463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d50ae-10a8-4b4c-8a8a-74433e5e65e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:45.982965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 258/538] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/role_assignments?user.id=e6c561ff3ab84f99b81eaccc9d9f2cad&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:45.984442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:45.984730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:45.985054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.002847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:46.003688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1684b-bef0-4ff3-bb35-0b6210a570bc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.004182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:46.042861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.043542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.043820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.044096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.044824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:46.045162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.047095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.059941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 281/539] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:46.060484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:46.070097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.094714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 259/540] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:46.135622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.177223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:46.188158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13794c-50ab-405a-b136-fde0ebb03ea4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.231758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.232926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.233797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.234277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.243323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.245566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:46.262007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:46.262886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:46.263944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 282/541] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:46.270031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.369152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ba73-0f47-490e-8fb2-e8bb7c6a3f02 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.386596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341695f9-0b8c-4ad4-a4b2-74a4851ad191 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.419570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.420433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.420734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.421089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.431817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:46.439774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:46.440657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:46.441729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 260/542] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:46.446485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.447282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.447647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.447924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.448723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:46.449055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.450500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.452833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.483363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 283/543] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:46.493817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.507902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:46.518687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:46.566274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0de3-7c6e-4331-b449-68583cfe2322 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.649805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.650850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.651365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.651709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.652556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:46.652942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.654401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.656116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c39ca-8526-4b7b-9340-21b8ccd5fa70 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.672774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 261/544] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:46.681982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:46.708249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.708943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.709232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.709507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.710351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=872c307ac6c24374b76b61b905d1f4e7) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:46.710665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.712183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.734310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 284/545] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/role_assignments?user.id=872c307ac6c24374b76b61b905d1f4e7&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:51:46.790496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f88b4-0e33-4d91-a713-8daee8b947a1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:46.800650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:46.827840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:46.828845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:46.829257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:46.829628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:46.841969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:46.843727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:46.863127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:46.864186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:46.865224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:46.865224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:46.865224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:46.865224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:46.866883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 262/546] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:46.874918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:47.001484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:47.028130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086a06-efcc-4113-9d93-d470d533f687 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:47.063082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:47.064096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.064597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.065104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.066016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:47.066515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:47.068098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:47.081578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 285/547] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:47.089093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:47.231787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19b43f-819c-49e7-8b98-b826fb1dab5a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:47.272345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:47.273186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.273469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.273747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.281673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:47.283147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:47.302839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:47.303827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:47.305284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 263/548] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:47.309952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:47.505358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d56031-dc99-41a8-b32f-8fdf7df3e504 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:47.550303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:47.550898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.551183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.551457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.552313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:47.552625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:47.554103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57313a94-f6be-452f-b18e-d3550b0af147 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:47.555849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:47.580598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 286/549] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:51:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:47.586058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:47.625065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:47.736860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696411f-d4d8-4a96-acb5-93971ffa4af4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:47.775360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:47.776027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.776309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.776581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.837132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a970c69-16c7-49fa-921f-f2129d7dd103 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.837319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a970c69-16c7-49fa-921f-f2129d7dd103 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.837600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a970c69-16c7-49fa-921f-f2129d7dd103 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.839243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 287/550] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:47.845108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:47.845399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:47.845681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:47.854601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:47.856616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:47.877707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:47.880360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:47.881528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.015592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 288/551] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:48.017057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 264/552] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:48.022444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.050343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.155387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c62a67-02fd-456d-ab9e-10fba5c11d08 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.188644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.200472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39114090-a32a-401b-b109-fcad11284361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.201317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39114090-a32a-401b-b109-fcad11284361 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.201627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39114090-a32a-401b-b109-fcad11284361 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.201902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39114090-a32a-401b-b109-fcad11284361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.212276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f16b-71ab-472a-95cd-e9faa60d7fae None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.212432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39114090-a32a-401b-b109-fcad11284361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.213869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39114090-a32a-401b-b109-fcad11284361 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-39114090-a32a-401b-b109-fcad11284361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:48.233127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:48.234248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:48.235102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:48.235102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:48.235102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:48.235102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:48.239270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 289/553] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:48.240874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353766c-0721-4184-a9a2-5303caad6f9b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.241185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353766c-0721-4184-a9a2-5303caad6f9b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.241465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353766c-0721-4184-a9a2-5303caad6f9b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.244058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.245012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 290/554] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:48.245877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353766c-0721-4184-a9a2-5303caad6f9b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.260834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.262181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.262468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.263167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.277463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8575b004-a14c-4411-b74e-1164c57f6b3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:48.288602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:48.289523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:48.290414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 265/555] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:48.291418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.293737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.293737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.313556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:48.315135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:48.363337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353766c-0721-4184-a9a2-5303caad6f9b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.412191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.412641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 266/556] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:48.412888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.413237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.413507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.413993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.414284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:48.414587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.415978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.435262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 291/557] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:48.438022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.485107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.538232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa292a-b9f4-44a3-8e1b-6feba620a3cb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.552417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.572324 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896137d4-6bff-496c-a061-14626e444d74 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.583987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.584807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.585234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.585638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.586553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:48.587173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.588539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.597486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 267/558] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:48.598476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.617677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.618598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.618883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.619230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.626154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.627669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f6a3ebbc-b0c7-457c-95d8-6b593fb6dc30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:48.642882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:48.644287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:48.645197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 292/559] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:48.646190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfa9c-bf59-467f-9a58-00ddd41e7240 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.646190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfa9c-bf59-467f-9a58-00ddd41e7240 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.646190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfa9c-bf59-467f-9a58-00ddd41e7240 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.647603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 293/560] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:48.652135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebfa9c-bf59-467f-9a58-00ddd41e7240 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.714927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df078a-32b7-4196-a94b-6be23bdd20bc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.752082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.754116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.755063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.755435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.755641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.760033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebfa9c-bf59-467f-9a58-00ddd41e7240 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.769227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:48.777104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:48.778309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:48.779325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:48.784377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 268/561] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:48.785012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.797629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.798579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.798794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.799065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.809437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.810945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.820751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-9d5e4fa2-76df-4516-9f85-72e096a674f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:48.830347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:48.831431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:48.832608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:48.832608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:48.832608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:48.832608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:48.838449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 294/562] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:48.838449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.838449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.838449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.852976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:48.854463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:48.922766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d318e1-de83-4eb4-b638-3a10eae20194 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:48.932879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 295/563] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:48.933662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:48.967863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:48.968136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:48.968382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:48.968658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:48.969502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:48.969811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:48.971269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:48.989960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 269/564] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:48.990789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.023508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.047920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337061ae-7435-47c8-8e9a-ddab888e42fc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.084077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.084756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.085071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.085316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.086096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.086402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.087787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.088446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.098681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 296/565] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.099312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.104111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb1999-45b4-4017-b4ac-3c1156b7cd41 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.141836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.142521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.142806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.143089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.143821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.144134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.145556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.160957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 270/566] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.160957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.216498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b618e4-9b93-4d5e-9034-3ff09a5ebea1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.251667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.252391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.252677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.253400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.260460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.264680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:49.265253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1b731-a932-4627-a539-6efde3bd1c45 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:49.273094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:49.274053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:49.274881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 297/567] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:49.275573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.300638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.301338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.301627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.301892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.302724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=ee9dc1ba0084428c9da313ebda0458e0) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.303036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.304488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.310044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 271/568] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/role_assignments?user.id=ee9dc1ba0084428c9da313ebda0458e0&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.310621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.326926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.391062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d29856-ead7-469d-9f1f-9860729aeb7b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.422724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248cf78-76da-4547-a14a-007f3ab76231 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.437110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.438175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.438604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.438993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.454067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.455574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.459290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.460107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.460386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.460655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.469454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.470930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c44766de-110c-45bf-864b-b1d11723799d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:49.476997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:49.477861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:49.478749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:49.478749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:49.478749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:49.478749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:49.478749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 298/569] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:49.479463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246c950-5931-42e9-9487-b369feddeee1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.479747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246c950-5931-42e9-9487-b369feddeee1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.480022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246c950-5931-42e9-9487-b369feddeee1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 299/570] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:49.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246c950-5931-42e9-9487-b369feddeee1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:49.491025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:49.492150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:49.493108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:49.497162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 272/571] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:49.497814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.538047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.582419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.590483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246c950-5931-42e9-9487-b369feddeee1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.612025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce10ce6-512a-455c-9dac-46076a7c2368 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.626903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.627634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.627921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.628195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.628975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.629285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.630669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c9446-d200-4fe6-9bf3-f6c1a23d3841 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.643950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 300/572] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.643950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.643950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.643950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.652000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.652670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.652933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.653293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.654014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.654403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.656588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.660866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:49.662119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:49.673348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 273/573] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.673348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.737490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 301/574] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:49.738617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.780521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47119-07af-4b6a-b72d-c2205c15fb65 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.798808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.821134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.821815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.822186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.822458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.823398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:49.823703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.825171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.827228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:49.850299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 274/575] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:49.851188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.857394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e7e50-41e3-48a4-a20f-6f163e381cde None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.892010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.892826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.893137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.893412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:49.903276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:49.904926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:49.922639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:49.923562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:49.924438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:49.924438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:49.924438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:49.924438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:49.924438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 302/576] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:49.924719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:49.955804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5626e8-0973-4094-bc4b-8749bdae2233 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:49.992188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:49.993918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:49.994343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:49.994783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.005169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.007359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.025054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:50.027219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.028474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 275/577] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.029430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.038122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc80fd-5d34-404d-abbc-47d21af30982 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.076302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.077678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.077959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.078232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.088615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.088827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:50.097441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.098403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:50.099233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 303/578] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.099910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.147201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560518-9930-416b-ae43-adac3a0d851a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.184110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.185030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.185359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.185614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.196555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.198142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.202357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfaabb-681d-4a00-8769-04639ccb2cd8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:50.217387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.218303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:50.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:50.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:50.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 276/579] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.219947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.242406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.243331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.243613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.243870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.245090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:50.245405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.246930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.264783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 304/580] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:50.264783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.280969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.331583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b59f4-5399-47c9-aeb0-dc8402fd3edf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.334919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.369706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.370387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.370674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.370776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc068-9d01-4b6d-9092-cb69af205c41 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.370951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.372739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:50.373213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.375326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.397608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 277/581] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:50.397608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.410855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.411551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.411820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.412090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.412860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:50.413197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.414629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.424482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 305/582] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:50.425710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.507714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.509160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd78db-4cd6-43bb-bf17-d5975dd84aad None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.533498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0001fe-baeb-49fb-ab45-31002934d0ab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.542878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.543551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.543832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.544104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.544817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:50.545156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.546707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.562314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 278/583] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:50.562314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.574543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.575603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.576060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.576372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.583832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.586473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.589257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:50.604821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.606335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 306/584] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.607282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.665475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e608-8ebb-4d88-b5c4-9fbec2184ef9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.699519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.700324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.700601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.700872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.711824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.712750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6c644-a456-4d77-98ba-aa30a740cb16 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.713300 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:50.732393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.733534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 279/585] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.734428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.753779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.754845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.755273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.755715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.758678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.774285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.775955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:50.801149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.802083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.802960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:50.802960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:50.802960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:50.802960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:50.803734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 307/586] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.804047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.815062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:50.840828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be1a2e-bf46-4153-a497-650c8c66d64a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.876585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.877544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.877889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.878224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.885490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.887059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:50.903230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:50.904139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:50.905245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:50.907094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 280/587] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:50.908056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:50.928546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7029-7c72-49a6-8f97-4b49f798be71 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:50.965838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:50.966888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:50.967503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:50.967872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:50.968784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:50.969117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:50.970846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:50.990160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 308/588] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:50.991112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.020079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c98af-e5af-4b6f-9f0f-a6aae4b6ce16 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.038914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.056870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.057573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.057855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.058128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.058852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.059159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.060546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.062207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.073007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 281/589] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.073882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.102721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03008-c5bf-4ad9-b33b-b90738eaa0e7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.139259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.139933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.140221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.140493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.141255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.141565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.142970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.158917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 309/590] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.158917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.191489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd2ec-9ae7-4f27-96ef-ac6879e2eb10 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.227074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.227753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.228032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.228308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.229129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.229443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.231261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.248832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 282/591] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.248832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.262913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ad75-7f27-4a5b-bf6a-b8775c365aee None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.270144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.298022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.298901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.299442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.299558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.308800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.310718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.312437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:51.332394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:51.333392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:51.334285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:51.334285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:51.334285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:51.334285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:51.334285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 310/592] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:51.335274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.360104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543266b-881f-4cd2-a107-6c2fe3ba98dc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.394348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.395144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.395422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.395687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.402253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.403908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:51.417531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:51.418413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:51.419290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 283/593] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:51.419882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.438878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e029f3-34c6-47d3-9341-03ec70338a1b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.472473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.473301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.473583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.473856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.481675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.483124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:51.497225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:51.498155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:51.499199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 311/594] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:51.499779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.518026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.527381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11298e-2d88-4f61-ac95-73bb33de2727 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.542979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.563613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.564518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.564804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.565190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.579851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=77cc74fa2a6c48d0b85e3b2e70f451f5, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.581322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10cd4d-a5a3-46c3-bf16-2c502388bbd1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.594416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 284/595] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:51 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:51.595758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b104-570c-4104-a33d-25015b2f9bac None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.596045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b104-570c-4104-a33d-25015b2f9bac None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.596316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0b104-570c-4104-a33d-25015b2f9bac None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.600963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 285/596] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:51.600963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b104-570c-4104-a33d-25015b2f9bac None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.601248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4918990-0328-4b11-bbc1-f96fb9949ba2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.641658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.642428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.642804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.643175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.644129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.644623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.646194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.664962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 312/597] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.664962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.726031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0b104-570c-4104-a33d-25015b2f9bac None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.748543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.770830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.771515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.771833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.772104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.772837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.773269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.774608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691ca1f-95bc-410e-b8f9-2007ca2c4dac None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.776896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14af5d9-e05a-446c-9e03-014301b4b210 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.792332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 286/598] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.797610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.798154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.798527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.813725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.814411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.814687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.814961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.815801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:51.816017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:51.816112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.817399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:51.817587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:51.836711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 313/599] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:51.837103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.837577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.896227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 287/600] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:51.896993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:51.950006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bed3f-4664-4278-b313-9ba63fc72e55 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:51.985595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:51.986504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:51.986791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:51.987066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:51.990115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:51.995197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:51.997512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.011341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1170e6-45a5-4fda-9d34-8a7b7bdc019c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:52.012538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:52.013433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:52.014400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 314/601] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:52.014898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.053650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.054835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.055105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/0f5e14d9d362461dae34b8330718002b/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.070717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=0f5e14d9d362461dae34b8330718002b, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.071842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.086029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 288/602] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:51 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/0f5e14d9d362461dae34b8330718002b/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:52.086889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.098614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.138795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf769379-e823-42ca-84b0-c139256c1d1b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.183141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.185113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.185563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.186541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.201443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.203522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:52.222716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:52.223902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:52.225037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:52.225037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:52.225037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:52.225037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:52.231604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 315/603] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:52.233056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.245251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff98ef-f7d9-4037-84a9-55b987b76db9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.288118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.289447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.289725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.290364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.292516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.305506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:52.321530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:52.323714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 289/604] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:52.324887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.359636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a14641-9b41-4f1b-a2ea-cdd0014bfa7c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.394690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.395213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.395980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.396360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.396775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.397813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:52.398223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.399813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.413950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 316/605] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:52.414972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.444968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2d395-dadc-4562-9f35-0ac5349399eb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.489161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.489834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.490136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.490943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.491680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:52.492239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.493913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.511812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 290/606] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:52.512675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.550152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0562b03-ef04-4dd7-ac84-386fb4b80d1d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.562324 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.596469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.597420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.597706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.598859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.599632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:52.599947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.601377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.613208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 317/607] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:52.614188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.644975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5ab92-0373-4815-b1f7-4b566e2e5454 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.681331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.688257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.689698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.690301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.690638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/e6c561ff3ab84f99b81eaccc9d9f2cad/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.712112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=e6c561ff3ab84f99b81eaccc9d9f2cad, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.714371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.745198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 291/608] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:52 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/e6c561ff3ab84f99b81eaccc9d9f2cad/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:52.746744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.757003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c40a5-ed46-4918-84cd-a96cc71bd834 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:52.799938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:52.800798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:52.801703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:52.801983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:52.810546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:52.812431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:52.831391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:52.833592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:52.834900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 318/609] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:52.841378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:52.880083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:52.966965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df9f85-fefe-4412-a7da-863a7e865240 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.011601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.012279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.012457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f180-a17e-4d93-b7bc-783ad7bc9cfd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.012632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.012817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.013651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:53.014561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.017813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.034385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 319/610] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:53.041869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.058935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.058935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.058935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.059584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.083072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.084941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:53.112691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:53.113686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:53.114561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:53.114561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:53.114561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:53.114561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:53.126417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 292/611] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:53.132924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.168419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:53.240509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:53.258349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a8710-db78-4d0c-b144-60d0284c25f1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.318854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4265d-8291-4874-b581-d7b6f914ed42 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.331465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.333223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.333721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.334232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/872c307ac6c24374b76b61b905d1f4e7/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.369867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=872c307ac6c24374b76b61b905d1f4e7, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.371343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.376835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.377943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.379087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.379087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.379640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:53.379981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.381581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.395145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 293/612] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:53.402111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 320/613] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:53 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/872c307ac6c24374b76b61b905d1f4e7/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:53.406127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.524393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:53.545150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b77a-c9fc-4bad-b1d3-ef27e4b7b269 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.597746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.598606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.598897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.599169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.612275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.613816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d53dae6d-718f-481a-9e06-74e12bb1c28c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:53.635817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:53.636835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:53.637816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 294/614] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:53.645235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.729229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4215a1-bf01-4d3d-9671-d913eda08ece None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.729575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4215a1-bf01-4d3d-9671-d913eda08ece None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.729784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4215a1-bf01-4d3d-9671-d913eda08ece None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.731780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 295/615] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:51:53.747619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.748222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.748222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.773698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:53.775128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:53.793374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f089d-eb13-4722-b808-c90f5bed16e8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:53.831033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:53.831776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:53.832053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:53.832370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:53.833255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:53.833560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:53.835000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:53.845281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:53.854005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 321/616] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:51:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:53.854428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 296/617] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:51:53.870373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.903051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:53.986326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170caf7e-3210-47f5-803a-84179e08b6bf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.022346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.023154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.023430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.023727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.034803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.036182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:54.054556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:54.055503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:54.056365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:54.056365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:54.056365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:54.056365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:54.056365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 322/618] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:54.064335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.065341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.109412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f11d95-468d-4798-acfa-caccf220fcb2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.157075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.175775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.176591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.176860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.177149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.191750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:54.199560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:54.200556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:54.201603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 297/619] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:51:54.208400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.249773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d409971-aa4b-4ff0-9ad3-9bc75d965423 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.301890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.302482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.302757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.303027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.303753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:54.304054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.305472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.324010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 323/620] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:54.329358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.383241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3327d0-1441-4f7b-87c6-21183b826817 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.420308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.421176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.421462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.421738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.422494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:54.422806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.424378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00eeace-15c5-429f-8661-5e84f264b3da None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.426657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.435887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 298/621] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:54.437640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ba4c-fc46-4e19-9091-b4218846954c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.438142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ba4c-fc46-4e19-9091-b4218846954c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.438574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37ba4c-fc46-4e19-9091-b4218846954c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.444290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 299/622] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:54.445401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ba4c-fc46-4e19-9091-b4218846954c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.446881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd5152-7500-4185-b04e-ae0179013c93 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.457745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.483094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.483903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.484178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.484447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.491589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.493069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ab981b89-df55-4b32-959f-af73a4ffb4fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:54.507388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:54.508353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:54.509680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 324/623] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:54.510377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.510476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.510757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.527688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:54.528881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:54.566500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37ba4c-fc46-4e19-9091-b4218846954c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.604322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 325/624] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:54.604322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.621916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.623124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.624457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.629070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.647645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.649660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:54.669555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:54.671162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:54.672042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:54.672042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:54.672042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:54.672042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:54.674106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 300/625] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:54.675083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.698957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.701828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cc435-88af-4a8e-adbe-fb44722b78e7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.737044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:54.744042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.745571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.746032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.746469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.747992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:54.748493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.750597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.766457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 326/626] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:54.767607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.811610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76f22a-04bb-46ba-b96a-899c62b5364f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.856060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.856796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.857784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.858102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.870996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:54.881902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:54.883052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:54.884080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 301/627] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:54.884891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.901803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21e4a-534c-46e2-87ee-034c2fe5d42d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:54.945504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:54.946422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:54.946772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:54.947114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:54.947946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:54.948348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:54.950003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:54.963896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 327/628] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:54.964953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:54.987755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.002740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10bf91-5eae-4995-a274-7fd8fed47b1b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.033333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.037861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.038887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.039284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.039769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/ee9dc1ba0084428c9da313ebda0458e0/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.055702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=ee9dc1ba0084428c9da313ebda0458e0, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.057618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.074629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfe675-efaf-45ed-a3b7-032fbbbcaab5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.077074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 302/629] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:51:54 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/ee9dc1ba0084428c9da313ebda0458e0/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:51:55.078834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.117737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.119314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.120794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.122646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.123984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:55.126563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.128276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.147016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 328/630] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:55.151976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.285031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.290269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94912875-0dac-41c2-8f7f-60e15bc16af1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.305604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5393c0-7c00-4e2a-b5d9-11d72fbae4c8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.339678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.341341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.342125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.342606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.348983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.349798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.350081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.350354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.353532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.355224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.361946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.363399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-59f8b679-38fb-4270-8d6f-bf3e4cae59f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:55.374619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:55.375548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:51:55.376498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:55.379149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 303/631] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:55.380667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4dd90-aed6-4532-bc0f-a2d8046abd6d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:55.381130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:55.381972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:55.383435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:55.383435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:55.383435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:55.383435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:55.383435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 329/632] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:55.383873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4dd90-aed6-4532-bc0f-a2d8046abd6d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.384301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4dd90-aed6-4532-bc0f-a2d8046abd6d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.384673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.390413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 304/633] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:55.390413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4dd90-aed6-4532-bc0f-a2d8046abd6d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.481465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.498821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496a070-f489-423a-b7a2-8a09f404f698 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.533206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4dd90-aed6-4532-bc0f-a2d8046abd6d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.535402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.536066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.536350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.536626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.537754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:55.537962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.539385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f954199-4ef1-4a13-974d-e5cad59a4a72 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.541245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.565120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 330/634] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:55.567447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.567934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.568865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.581010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.581774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.582205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.582566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.583368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:55.583789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.585404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.591520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:55.593739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:55.599050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 305/635] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:55.599992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.685035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 331/636] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:55.685035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.705506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.735968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85b3c-3d4b-4955-af07-df33df56108d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.774729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.775697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.776152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.776613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.788513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcda8e-a8c0-4f78-aa1d-5f1fbbb02d0d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:55.814509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.827490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:55.828315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:55.828593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:55.828863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:55.837074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.838773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:55.855698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:55.856711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 28 11:51:55.858041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:55.862013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:55.862473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 332/637] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:55.863165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.864369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:55.949447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:55.969801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 306/638] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:55.971022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:55.983482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbec56-947a-44d2-b8e0-e738373df587 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.021938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.023104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.023648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.024021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.038136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5e059ec4-623b-4b28-a49f-4ac98f0680ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:56.046608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:56.047543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:56.048366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 333/639] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:56.049569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58aaa2-f4c6-4e82-9c24-f8401f881fd9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.049728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58aaa2-f4c6-4e82-9c24-f8401f881fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.049946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58aaa2-f4c6-4e82-9c24-f8401f881fd9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.055154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 334/640] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:56.055154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58aaa2-f4c6-4e82-9c24-f8401f881fd9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.083427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a143-d0ec-41a5-a231-36f3b6a99071 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.098672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.119996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.120668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.120970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.121242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.122073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:56.122400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.123860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.144380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 307/641] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:56.145272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.161096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58aaa2-f4c6-4e82-9c24-f8401f881fd9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.198478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.200247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.202114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.202570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.203174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.215330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.216802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2fd0d34b-115f-4719-a14e-c6a0fd3d26ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:56.236342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:56.237570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 335/642] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.243619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.257438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:56.258639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:56.271779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045d640-2257-42f3-98f8-20bb0d0f6b0b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.309762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.310457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.311753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.311753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.312558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:56.312870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.314330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.327401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 308/643] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:56.329876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.340282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 336/644] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:56.341300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.368375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.446062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.447392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552687c6-c4d8-4368-a04e-65c9844d4b76 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.449677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2cdb5-e621-47a1-8e1c-4e84980e68d2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.484510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.484830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.485134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.485419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.485514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.485695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.485803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.485966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.486427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:56.486733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.488112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.506620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 337/645] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:56.507534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.554971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.556478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.594995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.614904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28d270-6ed5-4d76-9624-83665c497053 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.651294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.652377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.652651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.652955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.654133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.658768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 309/646] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:56.659931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.668803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.670419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:56.687908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:56.688828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:56.689942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:56.689942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:56.689942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:56.689942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:56.689942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 338/647] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:56.690916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.774884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917e8f7-ba73-4dbf-a18a-497f861cbe1a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.797658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27302e-6116-4457-92e9-20893a7bd2c1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.811305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.812119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.812415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.812660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.823620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:56.831344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:56.831507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:56.832396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:56.833628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:56.834460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.834460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.834460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.834659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 310/648] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:56.834659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.840541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.845394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.847489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:56.863360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:56.864467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 339/649] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:56.866060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:56.904792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:56.949062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30537735-5c36-4140-859b-1e0bde0c2f97 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.976530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb93ed-19dc-4f6d-93d9-df43ea4a062f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:56.981785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:56.982588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:56.982943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:56.983207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:56.993800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:56.995258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:57.012976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:57.013925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:57.014822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:57.014822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:57.014822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:57.014822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:57.015576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 311/650] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:57.016357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.018574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.019249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.019531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.019816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.020801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.021939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.023364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.041190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 340/651] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.042028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.101582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.128743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09fbe-c276-4c18-8907-4ef85d9c6eab None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.143227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.151013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6b96-899d-424c-817b-9b3df6dbab8e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.165199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.165880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.166161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.166435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.167186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.167521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.168897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.177572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 312/652] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.178361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.191513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.192470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.192832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.193144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.193945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.194258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.195706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.213996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 341/653] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.214888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.280660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b446237-d6dc-48b0-b904-5b45e8e1bb26 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.314209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ee749-b41c-4517-9aa1-ee7e98654c15 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.314537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.315199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.315478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.315748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.316461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.316760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.318171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.330984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 313/654] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.331483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.332106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.353134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.354174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.354451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.354722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.362349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.363817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.376979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:57.380157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:57.381288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:51:57.382470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:57.386237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 342/655] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:57.386940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.451160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df251c5-c34b-4397-b647-b6e9e05d0bdf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.486233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.486791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892840e4-d045-42cf-b661-1d52d0f1ae68 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.487024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.487304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.487569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.497718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.499796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-00c11936-0227-46c4-ac3e-b5c9a8560b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:57.518840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:57.519735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:57.520607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:57.520607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:57.520607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:57.520607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:57.520607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 314/656] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:57.520876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.520876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.521406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.521475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bceb1-3b9a-4a4a-b0a1-9a44813aa25b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.521543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.521619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bceb1-3b9a-4a4a-b0a1-9a44813aa25b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.522155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bceb1-3b9a-4a4a-b0a1-9a44813aa25b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.522312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.522614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.524939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 315/657] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:57.524939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bceb1-3b9a-4a4a-b0a1-9a44813aa25b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.525140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.532101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 343/658] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.532776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.578347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.625047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.643418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bceb1-3b9a-4a4a-b0a1-9a44813aa25b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.650705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5a18-c828-4e81-9f99-052cdc430b4f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.685791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.686945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.687226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.687497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.689264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.690073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.690349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.690617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.691475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.691787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.693295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7dc3b-03db-41f6-b35e-181bb6b0d4f6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.697098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.698843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:57.713673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:57.714618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:57.715496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 316/659] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:57.716536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.722041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 344/660] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.723474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.723790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.724086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.740883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:57.742117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:57.829091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 345/661] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:51:57.830017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.843616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9b1ee-23ac-48a9-ae9a-43172bdf4a10 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.860253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.881811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.882579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.882865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.883139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.883863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.884166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.885583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:57.898228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 317/662] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:57.898838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:57.908792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:57.943353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73126b-9a6b-4492-9afe-ddb2e9e9d61d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:57.979408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:57.980836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:57.981204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:57.981548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:57.982402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:57.983013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:57.984421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.001320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 346/663] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:58.001320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.008637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aae3d-22a2-4fd0-af78-6d7c1a920f02 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.045865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.046668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.046946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.047223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.096593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.111153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a3c9c-8ffe-45c9-8c23-f1e51bb7ba68 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.129219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.130758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.139985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.147282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.148086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.148368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.148631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.158814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:51:58.166414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:58.167526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:51:58.168560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:58.172943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 347/664] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:58.173797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.242054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 318/665] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:58.242906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.284749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d032d-39c3-4a44-b4cc-4da5b1261bd4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.320215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.321050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.321330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.321841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.330017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.331501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:58.345729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:58.346776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 28 11:51:58.347752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:58.348305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2dc4e-e01e-406e-acae-f92ec0248986 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.351453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 348/666] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:58.352287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.361587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.364942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.392079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.392744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.393106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.393400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.394241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=0f5e14d9d362461dae34b8330718002b) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:58.394553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.396002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.402039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 319/667] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/role_assignments?user.id=0f5e14d9d362461dae34b8330718002b&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:58.402812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.502385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80e70c-b50f-4596-b86e-ac0e74be0c5b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.518468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71606-dc00-419b-86b7-aa642228a27d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.544600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.546451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.546952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.547540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.548527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:58.549109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.550910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.555290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.556108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.556387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.556660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.562357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 349/668] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:58.563516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.569576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.571183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:58.590727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:58.591676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:58.592611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:58.592611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:58.592611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:58.592611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:58.592611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 320/669] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:58.594062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.625569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.653921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.703591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a501bd-d4fe-4efa-bf01-032aaf5e183c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.728946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b43c58-7652-446b-8217-703de52be3c1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.752873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.753608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.753891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.754170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.755769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:58.756107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.758145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.771866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.772953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.773429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.773891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.783802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 350/670] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:58.784685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.788634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.790464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:58.809501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:58.810461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:58.811410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:58.811410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:58.811410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:58.811410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:58.813703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 321/671] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:58.814725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:58.893454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.910690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d87527-89ac-46c0-b37b-87d41290071b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.921959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926da3a-9522-45f1-968e-a3d580cfda8f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:58.944103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:58.955136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.956080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.956374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.956636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.957502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:58.957813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:58.959432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:58.960754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:58.961798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:58.962272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:58.962765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:58.973321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 351/672] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:58.974243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.039548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.041443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.117220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856879d-4b23-4958-bdf3-4c184e2588fe None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.120836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.149666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 322/673] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:51:59.150820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.163610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.164314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.164840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.165149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.165887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:59.166203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.168295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.185397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 352/674] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:59.186352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.245383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.276559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62394a72-06b5-4bd1-befc-ca81ee4984a9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.300434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec910506-c403-4a8e-9147-c6ec81170e01 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.322640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.323467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.323750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.324027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.334224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.335706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.337860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.338904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.339402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.339881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.350793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:59.351921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:59.352785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:59.353734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 323/675] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:59.354683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.356098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:51:59.375286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:59.376229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:59.377152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:51:59.377152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:51:59.377152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:51:59.377152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:51:59.379124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 353/676] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:59.380208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.400112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.483736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.487636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a9f1d-9947-4da3-b3df-06aa978a8927 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.493399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f9e6-984c-42a1-a020-db299cd3cdf4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.524626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.525785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.526262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.526746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.527782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:59.528282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.530047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.538012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.538831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.539114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.539391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.543889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 354/677] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:59.545154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.548924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.551390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:51:59.570243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:59.571247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:59.572163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 324/678] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:59.572762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.671752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4dfc1-2903-4706-af67-f773e57d1d6d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.706295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.712577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.713283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.713564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.713836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.714176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9baf-b62c-4073-ae31-b6d8a5a88c1a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.714569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:59.714870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.716239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.728993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 355/679] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:59.730061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.754533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:51:59.755954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.756614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.756894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.757191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.758025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=e6c561ff3ab84f99b81eaccc9d9f2cad) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:59.758332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.759829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35d589-e018-4122-b9a4-541c208f926b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.773020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 325/680] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/role_assignments?user.id=e6c561ff3ab84f99b81eaccc9d9f2cad&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:59.773020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df7b2-eb80-4409-b74c-22c66433ba9b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.773020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df7b2-eb80-4409-b74c-22c66433ba9b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.773020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df7b2-eb80-4409-b74c-22c66433ba9b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.774626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 326/681] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:51:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:51:59.775460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89df7b2-eb80-4409-b74c-22c66433ba9b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:51:59.890031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a37a9-5e44-4c50-9423-e2ef66307977 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.905943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89df7b2-eb80-4409-b74c-22c66433ba9b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:51:59.929531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.930435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.930718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.930989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.931832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:51:59.932136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.933604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75b366-d160-43c2-a62b-dc0d959aaf0a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.953050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592047df-3a80-4077-984e-27064c03420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:51:59.954119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592047df-3a80-4077-984e-27064c03420e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.954252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592047df-3a80-4077-984e-27064c03420e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.955372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592047df-3a80-4077-984e-27064c03420e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.958931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 356/682] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:51:59.960373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:51:59.960653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:51:59.961900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:51:59.968090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592047df-3a80-4077-984e-27064c03420e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:51:59.969856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592047df-3a80-4077-984e-27064c03420e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:51:59.976685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:51:59.977912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-592047df-3a80-4077-984e-27064c03420e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:51:59.991748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:51:59.992646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:51:59.993565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 327/683] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:51:59.999749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592047df-3a80-4077-984e-27064c03420e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.023130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.072932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 357/684] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:00.072932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.086515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.149582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592047df-3a80-4077-984e-27064c03420e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.185308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.185948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.186586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.186667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.220899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c1b34-64da-482d-a662-fc41c6c568aa None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.266952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.268016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.268759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.269147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.269455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.269577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.270336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:00.270703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.272202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.289955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 358/685] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:00.289955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.299667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.370200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.409954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecced6-96e8-4a87-b674-544b325cf23d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.421254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 328/686] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:52:00.422161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.459575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.460776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.461483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.462008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.474040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:00.482385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:00.484101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:00.485393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:00.492484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 359/687] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:00.493641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.532207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0ed7-358e-4f80-985c-a222024098af None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.549694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.568877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.570261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.570261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.570261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.570967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=872c307ac6c24374b76b61b905d1f4e7) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:00.571277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.572727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.584936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 329/688] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/role_assignments?user.id=872c307ac6c24374b76b61b905d1f4e7&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:00.584936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.624591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc257861-6ecb-41e3-985a-0d39e48442b9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.677896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.678726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.679016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.679297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.684007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.694564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03406-c819-4079-a548-861d7dd44c98 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.699576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.701380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-505a98d2-2556-4be1-ae79-29238ccda443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:00.730892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:00.731824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:00.733334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:00.733334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:00.733334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:00.733334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:00.733334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 360/689] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:00.734631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b233d9-6cfd-4383-9790-0ca21dd8982b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.734989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b233d9-6cfd-4383-9790-0ca21dd8982b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.735335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b233d9-6cfd-4383-9790-0ca21dd8982b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.738465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.739371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.739657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.739927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.740710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:00.741006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 361/690] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:00.741110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.742458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.742862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.743284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.743571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.757662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 330/691] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:00.758575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.769580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:00.770863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:00.828180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:00.856668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 362/692] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:52:00.857697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.906416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86c60-214b-4a36-b3d7-65a16e9aea3b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:00.950143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:00.951900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:00.952186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:00.952770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:00.953587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:00.954159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:00.956656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:00.975589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 331/693] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:00.976735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:00.989053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7f867-82d6-4b2b-833b-f186a1338484 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.000757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.028100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.029086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.029514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.029876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.044792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.047010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:01.071756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:01.073344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:01.074798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:01.074798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:01.074798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:01.074798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:01.076990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 363/694] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:01.077922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.100440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088db42-337b-489f-b397-b49b68f1e75a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.103875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.138040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.139129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.139508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.139871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.150649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.152240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:01.173061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:01.174003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:01.174870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:01.174870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:01.174870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:01.174870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:01.174870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 332/695] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:01.178508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.195698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07446c-ab1b-40aa-b482-f75a258fa9ca None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.232106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.233456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.233740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.234011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.241405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.242796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:01.256978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:01.257875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:01.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:01.262992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 364/696] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:01.264045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.298095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b071207-aa78-48d3-805a-b9a5ff908eb4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.298582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.333963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.334696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.334909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.335192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.335908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:01.336216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.337628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.349938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 333/697] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:01.350874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.376437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.398272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fce820-edf8-4e76-b5a9-b5f8b7774ddf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.433783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.434514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.434883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.435425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.436036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:01.436718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.455446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 365/698] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:01.455446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.486685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5bb8b-d3db-4535-a784-9e267ea5a10f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.522080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.522868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.523212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.523552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.524411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:01.524785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.526394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.544575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 334/699] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:01.545669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.555014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.580705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d2b70-8f89-4f5e-abf9-81dba5ae99da None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.619393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.620338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.620685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.621064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.626296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.629709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.631677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:01.648591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:01.649777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:01.650930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:01.651636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 366/700] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:01.652500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.654834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476db0c-4fc2-4f51-9ee6-eae5ed30f5cf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.689964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.691045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.691516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.691975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.692882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.694553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.721210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 335/701] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:52:01 2022] POST /identity/v3/users => generated 254 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:01.722593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:01.762652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d6f1-2d89-457f-aa12-3f6ee84fb78a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.815538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:01.816544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.817041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.817487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.818689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=ee9dc1ba0084428c9da313ebda0458e0) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:01.819197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:01.820842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d85928-59df-4f09-8580-47a643677b8d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:01.828702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 367/702] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/role_assignments?user.id=ee9dc1ba0084428c9da313ebda0458e0&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:01.908493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4274-99b0-4e73-9378-ecd49001ce0d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.909284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4274-99b0-4e73-9378-ecd49001ce0d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.909860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4274-99b0-4e73-9378-ecd49001ce0d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.912100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 368/703] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:01.916107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb4966-fb69-41d3-a9be-e73cc694cb86 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:01.924039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:01.924387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:01.924599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:01.952666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:01.953942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:01.955209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:01.998297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:02.006359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.011580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.011897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.012131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.013049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:02.017423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.018953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.042641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 369/704] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 28 11:52:02.051883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 336/705] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:02.057276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.069864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.183930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14189b-a04d-4575-a408-b8d6586302e8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.197808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d156b2-16aa-4c27-81b6-f0ca98cfa1f5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.234848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.235523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.235803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.236081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.241352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:02.243422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.244304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.244590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.244854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.245549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.246937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:02.267660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:02.268711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:02.273735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 337/706] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:02.277702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.342463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:02.350306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.351809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.389562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f37b56-a587-4be3-9dd3-7e4046f06a07 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.425198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.425881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.426161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.426439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.427218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:02.427527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.428993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.434190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 338/707] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:02.440043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.551697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0a7b-ea71-4f8b-b04b-e6730a92589d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.577294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:02.589113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.590103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.590516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.590889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.591707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:02.592177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.593887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.600438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 339/708] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:02.610458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 370/709] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:52:02.618196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.751897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a81e8a-e3f5-42d3-9cd5-725f8da5b7d0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.791380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.792202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.792481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.792749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.809037 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.810627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-857037b0-30c4-4290-a0b7-6ae170c8feb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:02.829082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:02.831123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:02.832358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:02.832358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:02.832358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:02.832358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:02.832358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 340/710] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:02.836270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:02.875721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:02.943887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20101d1b-05ce-4ee6-a948-3300fa6816a7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:02.986252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:02.986937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:02.987282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:02.987556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:02.988354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:02.988658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:02.990120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.001747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33cb1e-13c6-4363-bedf-0cfe503cdb2a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.002294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33cb1e-13c6-4363-bedf-0cfe503cdb2a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.002294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33cb1e-13c6-4363-bedf-0cfe503cdb2a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.003950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 341/711] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:03.006315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 371/712] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:03.009030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.009382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.009660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.012000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.049943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:03.051145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:03.124173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:03.154795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:03.170961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986787-9b16-4340-8fc2-19b6c2a3378c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.173023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 342/713] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:03.194972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.213361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.214165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.214432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.214705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.221996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.223540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:03.241566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:03.242545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:03.243527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 372/714] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:03.249789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.357677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bda81-e129-4bd7-9bab-149d46bd915d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.395033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bb275-d876-4d4d-b218-61ab3a6a7d4d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.395381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.396052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.396328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.396598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.404991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:03.404991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.404991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.416961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 373/715] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:03.420494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.442284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:03.476853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.478067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.478550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.478991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] PATH_INFO: `/v3/services/network` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.479824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.481832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:03.512126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:03.513308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:03.514220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 343/716] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/services/network => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:03.519947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.554945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f8bc7-6441-49b1-a011-ed7a84fad909 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.584160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:03.601085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.601750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.602029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.602300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.603648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:03.603962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.606620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.613761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 374/717] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:03.740133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb0683-702c-4f03-b5eb-af898032f96d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.790307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.791018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.791296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.791564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.792294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] RBAC: Adding query filter params (name=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:03.792599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.794140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972172b5-bb60-45dd-bf99-ac59bc2238fa None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.799815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 344/718] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:03.805482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:03.899219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:03.928515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5e770-3489-46b6-93c1-945f6417c082 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:03.971422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:03.972095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:03.972380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:03.972656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:03.973390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] RBAC: Adding query filter params (type=network) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:03.973691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:03.975066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008e342-b6c4-48d8-bd1e-4aa3ee7dfc77 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:03.985967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 375/719] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:04.217097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ac306-d2d5-4e8c-9914-8371578a26c6 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.217646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ac306-d2d5-4e8c-9914-8371578a26c6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.218312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ac306-d2d5-4e8c-9914-8371578a26c6 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.220139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 345/720] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:04.226239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.226547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.226796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.249018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:04.250260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:04.323104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 376/721] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:04.348581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ac306-d2d5-4e8c-9914-8371578a26c6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:04.451536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:04.466526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ac306-d2d5-4e8c-9914-8371578a26c6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:04.508272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:04.509045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.509322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.509590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.522542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e55e72ba-54ab-4f1a-a181-353c1699ea37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:04.537095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:04.538059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:04.538894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 346/722] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:52:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:04.544514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:04.631003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:04.659685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69185f04-d880-4f16-85de-b07da91b2d20 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:04.710065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:04.710748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.711029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.711304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.712058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:04.712365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:04.713779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:04.718043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8c03b-27dc-4eec-8b21-ae5623de9334 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.718324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8c03b-27dc-4eec-8b21-ae5623de9334 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.718657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8c03b-27dc-4eec-8b21-ae5623de9334 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.720261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 347/723] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:04.725563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.725843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.726115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.727365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 377/724] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:04.735929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:04.748714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:04.749865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:04.879916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd6d67-b090-4fef-8c5e-3a27ddf28634 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:04.894597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 348/725] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:04.922139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:04.924413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:04.925828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:04.926303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:04.926783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:04.934466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:04.948367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:04.952421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:04.978795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:04.979843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:04.981121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:04.981121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:04.981121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:04.981121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:04.984466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 378/726] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:04.990962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.111865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.140052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f745d-142e-4ba2-a5bf-5549603e1c90 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.163254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5054a-a195-4535-adba-68265ac92c59 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.215194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.216411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.218247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.219189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.232350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.233691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.234169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.234630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.235540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:05.236033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.237736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.251880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 379/727] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:05.256778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.257565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:05.277297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:05.278282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:05.279111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:05.284366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 349/728] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:05.287997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.339822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.397896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ef002-cb79-4930-a29c-30f5fb834adb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.408645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf457-0e9b-41d1-b93f-cc324428d4db None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.444364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.445409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.445837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.446256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.447357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:05.447840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.448372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.449336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.449743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.449876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e31347-a54c-4956-a750-1d4ea6a2e845 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.450196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.458277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.459849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.460762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 350/729] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:05.463185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d6be7-42a7-426a-83cb-533dd0199469 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.463650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d6be7-42a7-426a-83cb-533dd0199469 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.464068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d6be7-42a7-426a-83cb-533dd0199469 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.465914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 351/730] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:05.466754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d6be7-42a7-426a-83cb-533dd0199469 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.476474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f2d2477d-424a-4d90-b778-4ea37365c6c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:05.479961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:05.481305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:05.482238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:05.486148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 380/731] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:05.488024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884897-af33-4d01-88a0-48d328de783a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.488480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884897-af33-4d01-88a0-48d328de783a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.489253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5884897-af33-4d01-88a0-48d328de783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.511125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a5884897-af33-4d01-88a0-48d328de783a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:05.512805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a5884897-af33-4d01-88a0-48d328de783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:05.599352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 381/732] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:05.600615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5884897-af33-4d01-88a0-48d328de783a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.607495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d6be7-42a7-426a-83cb-533dd0199469 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.643906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.659022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.659766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.660051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.660325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.672020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.673443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:05.691582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:05.692531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 352/733] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:05.694024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.719247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5884897-af33-4d01-88a0-48d328de783a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.753531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.755684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.756577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.756997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.757426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.758487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:05.758684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.760218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.784291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 382/734] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:05.784291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.815911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b94c-9c39-4808-a677-12649c5ee829 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:05.852514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:05.853562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:05.854224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:05.854682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:05.856007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:05.857656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:05.879057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 353/735] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:05 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:05.880161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:05.899946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:05.916396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba24dd-cd51-42a0-bb03-6589cd358c33 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.009700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.012660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.013372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.014052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.015436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:06.016937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.020586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3717c-79b7-498c-9cf0-be457045a248 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.030634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6de4-b26e-4685-b2bb-ee1c972caad3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.050754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.054741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 383/736] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:06.057921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e06cf-8516-4efd-8e98-6f66a621cdf5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.058574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e06cf-8516-4efd-8e98-6f66a621cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.059393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e06cf-8516-4efd-8e98-6f66a621cdf5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.062580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 384/737] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:06.065575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e06cf-8516-4efd-8e98-6f66a621cdf5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.074149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.074851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.075167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.075442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.175854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.177857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.217454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e06cf-8516-4efd-8e98-6f66a621cdf5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.255207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.256014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.256303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.256573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.265146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.266601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ed68331b-16ac-4998-84c0-222f526c1109 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:06.282386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:06.283389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 385/738] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:06.285403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.286502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.286502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.304463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:06.305782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:06.315151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.325020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 354/739] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:52:06.325896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.354347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.396505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 386/740] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:06.397652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.496768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d15182-b4a6-43f7-9280-c59a2d70229e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.520598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b433cf-11c5-42c1-800c-b3e35078ff1c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.544515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.545421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.545700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.546053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.546994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:06.547204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.548650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.560016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.561476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.561769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.562040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.571555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 355/741] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:06.572352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.573797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.575269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.577676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5b8f4072-c0a1-483e-bc8e-a5dd48820413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:06.595661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:06.597804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 387/742] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3de2c-aad6-4257-998e-d52c75a07e0a None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3de2c-aad6-4257-998e-d52c75a07e0a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.599464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3de2c-aad6-4257-998e-d52c75a07e0a None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.600273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 388/743] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:06.601109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3de2c-aad6-4257-998e-d52c75a07e0a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.660733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.688533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f92-2c91-4bcb-be89-71516eb9a941 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.716499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3de2c-aad6-4257-998e-d52c75a07e0a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.725567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.726402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.726681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.726953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.739685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:06.747376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:06.748286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:06.749132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 356/744] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:06.749829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.760272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.760931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.761243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.761512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.840266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.848879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.850466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.891377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b69667-d6a5-4233-b442-9fd35ad0b5bf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:06.932795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:06.933919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.934407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.934781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.936197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:06.936307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:06.938390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb708a5-2387-4f10-8ffd-097c60b4e246 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:06.940272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:06.953585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 357/745] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:06.955092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:06.955371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:06.955638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:06.970498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 389/746] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 28 11:52:06.971666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:06.975911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:06.977178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:07.056261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 358/747] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 28 11:52:07.058052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.077489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.081813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878e660-815f-4cfe-bed6-d5c023c90009 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.130935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.132244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.133277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.133552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.134310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:07.134646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.136022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.152248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 390/748] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:07.152248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.175188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0425cb-0dd2-4c91-8230-4705d776c195 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.205894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.218904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.219950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.220430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.220879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.229662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.231568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:07.250719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:07.251637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:07.252538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:07.253687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 359/749] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:07.253687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.262227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d3924-b4d4-4d36-86da-8d686602c899 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.300126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.300954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.301318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.301532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.313557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.314941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:07.333070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:07.334020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:07.334971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:07.334971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:07.334971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:07.334971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:07.334971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 391/750] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:07.335249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.347415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.369431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae07f4-b39f-4cee-9806-c3410b0d1687 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.407164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.407842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.408192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.408455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.409216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.410598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.432941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 360/751] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:07 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:07.432941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.449201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4b296-33eb-4049-9c11-5feeede376ab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.494093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.500843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.501791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.502157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.502542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.516146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.517825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:07.537345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:07.538334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:07.539234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:07.539234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:07.539234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:07.539234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:07.543859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 392/752] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:07.544879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.558134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9165f-76d5-4a8b-a8d1-8f69c974d659 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.596353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.596974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.597256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.597523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.598318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:07.598620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.600078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.608385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.614856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 361/753] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:07.615921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.663765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f55132-0032-4116-80d7-a8967e5dd39e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.716958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.716958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.716958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.716958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.717563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:07.717563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.717563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.729146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd9f27-d415-48d2-9320-d55cc5732480 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.732932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.734922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 393/754] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:07.736021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.776189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.776733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.777103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.777462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.778255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:07.778626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.780452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.796149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 362/755] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:07.801696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.874971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77cac8-73ac-4314-9a05-710750fbf88d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.898623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:07.909513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04205-065c-4115-a075-90873e31ff28 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:07.924667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.925656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.925778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.925988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.926882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=81b68417e9e043b3abc779986564daff) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:07.927208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.928650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.934871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 394/756] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/role_assignments?user.id=81b68417e9e043b3abc779986564daff&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:07.935737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:07.953976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a063-9159-4901-911a-5965272fe077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:07.955299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a063-9159-4901-911a-5965272fe077 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:07.955957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a063-9159-4901-911a-5965272fe077 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:07.956366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520a063-9159-4901-911a-5965272fe077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:07.966859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520a063-9159-4901-911a-5965272fe077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:07.969961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520a063-9159-4901-911a-5965272fe077 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2520a063-9159-4901-911a-5965272fe077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:07.991802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:07.993041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:07.993988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:07.997048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 363/757] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:08.005165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a063-9159-4901-911a-5965272fe077 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.055711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:08.097653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738530c-72eb-4c64-b1e9-ccf34e9eefcc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.182941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.183679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.183959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.184229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.195173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.196793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.211854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520a063-9159-4901-911a-5965272fe077 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d5f3cbcb-d421-40c5-8dd3-dbca157ef803 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:08.213741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:08.214677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:08.215634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:08.216665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 395/758] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:08.222096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f340-3835-457c-8406-fd413c5f14e2 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.222387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f340-3835-457c-8406-fd413c5f14e2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.222672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f340-3835-457c-8406-fd413c5f14e2 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.224795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 396/759] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:08.226029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f340-3835-457c-8406-fd413c5f14e2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.267539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.268475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.268869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.269290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.271720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:08.272192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.274347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c513a-861c-4a36-9eb1-00d424cb50d8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.278963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:08.294388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 364/760] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:08.295811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.296053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.296329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.327712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:08.328917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:08.377177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f340-3835-457c-8406-fd413c5f14e2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.421895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.422586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.422863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.423140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.424097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:08.424454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.425956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.429065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 365/761] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:08.429974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.460572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 397/762] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:08.462369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:08.465702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.567253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:08.571105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e848ad-6f94-455f-980e-2f688ac15163 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.578879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709fb0f-95a8-41a7-9401-0722691f83a0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.607495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.608540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.608928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.609312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.610066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.611552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.621261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.621979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.622259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.622531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.623459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=ee9dc1ba0084428c9da313ebda0458e0) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:08.623957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.625431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:08.626842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:08.628125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:08.629955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 398/763] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:08.631977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.637257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 366/764] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/role_assignments?user.id=ee9dc1ba0084428c9da313ebda0458e0&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:08.638078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.742967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:08.771301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67bb50-ed95-43d9-9d42-81472fa2ade7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.834830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.835534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.835816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.836090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.836830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:08.837169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.838796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.845524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 367/765] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:08.852870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:08.909237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e8c9e-b830-4581-bc1a-7174637de6ca None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:08.956071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:08.956890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:08.957200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:08.957472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:08.968774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:08.970271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:08.992590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71588b32-f23c-46a5-89ff-4af44eda62d4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:09.003027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:09.006027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:09.013063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:09.013063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:09.013063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:09.013063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:09.013063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 399/766] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:09.014109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.046792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.047478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.047757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.048027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.048740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:09.049068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.050452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.054927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:09.061663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 368/767] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:09.065410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.078459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:09.208743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8752f3-fffc-4d63-97e5-2de1a8ff52e0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.256295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.256919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.257209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.257494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.258225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:09.258541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.259933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.266838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcc911-1409-4543-b055-d6b53246b7dc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.284153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 400/768] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:09.289955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.322184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.322871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.323149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.323414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.324211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] RBAC: Adding query filter params (interface=public, service_id=664d41b550e046d0b9476304313200f2, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:09.324518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.325937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.345569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 369/769] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:09 2022] GET /identity/v3/endpoints?service_id=664d41b550e046d0b9476304313200f2&interface=public®ion_id=RegionOne => generated 194 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:09.351457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.420867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:09.470279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1791df7-8d0b-424e-8e85-2e8cbc1401f7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.475745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:09.535732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693cc85-0bdf-4710-989d-c1abd90cea07 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.537952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.539218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.540013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.540740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.551170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.553558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:09.571333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:09.572290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:09.573210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:09.576003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 401/770] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:09.582137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.597859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.598562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.598840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.599107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.599797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.601228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.616853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 370/771] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:09 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:09.739911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7bebf-7155-41e5-a555-e68281c35cda None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:09.794045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:09.794652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:09.795012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:09.795284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:09.796114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:09.796423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:09.797954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c01512-ebaa-4d8c-9beb-151ac995b6d7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:09.799793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:09.813940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 402/772] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:09.821494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:09.959386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefe09-91f4-4204-89ac-56e2a9319d66 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:10.013499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198414-168a-4dc6-82bf-fb26893d4d1b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.013891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198414-168a-4dc6-82bf-fb26893d4d1b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.014169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198414-168a-4dc6-82bf-fb26893d4d1b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.015585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:10.015888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 403/773] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:10.016972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.017351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.017734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.020551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=77cc74fa2a6c48d0b85e3b2e70f451f5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:10.022174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.022459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.022630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:10.022725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.025899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:10.045154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:10.045487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 371/774] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:09 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:52:10.050116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:10.133598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 404/775] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:10.178642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.306387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef4e3d-e916-490f-830d-503ea280e9c2 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:10.342537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:10.342965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.343241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.343510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] PATH_INFO: `/v3/services/network` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.344169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:10.345571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-19e41cf1-5535-49ec-997a-66f4980b804c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:10.358278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:10.360608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:10.362680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 372/776] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:10.368313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.518770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c2560-d05a-436b-8d77-becd0485c265 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:10.552377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:10.557128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682a5a9-64e5-4ede-b9a1-a139ef8c80a3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.557582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682a5a9-64e5-4ede-b9a1-a139ef8c80a3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.557677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682a5a9-64e5-4ede-b9a1-a139ef8c80a3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.559241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 373/777] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:10.564301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.564582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.564855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.573609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:10.574640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.575114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.575586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.576808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] RBAC: Adding query filter params (name=network) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:10.577376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:10.578959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:10.582795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:10.584375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:10.587129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 405/778] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:10.593982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.650520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:10.666044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 374/779] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:10.744894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.788215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e4e93-4da6-47e3-bfcb-119e8b877e1f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:10.842077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:10.842736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.843021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.843296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.844026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] RBAC: Adding query filter params (type=network) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:10.844332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:10.845748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:10.859843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 406/780] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:10.866847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.884115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8df3-7fd0-474b-afcc-1594cca0416b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:10.917965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:10.918773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:10.919045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:10.919309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:10.929909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:10.940680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:10.941690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:10.942527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 375/781] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:10.949342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:10.985000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.043430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.074245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11062f72-5875-4b6b-a8c8-3d5828612546 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.133160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a25987-a698-414a-98e0-7783079ba7df None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.137328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.138254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.138710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.139122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.140470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] RBAC: Adding query filter params (interface=public, service_id=0a285bf66cb64c4cb6ec19b204a17688, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:11.140955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.150986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.171788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.172558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.172846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.173145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.173906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:11.174217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.175611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.176433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 407/782] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/endpoints?service_id=0a285bf66cb64c4cb6ec19b204a17688&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:11.177624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.194334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 376/783] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:52:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:11.201106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.348049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436778-2597-4dd6-9b93-847c49976900 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.370851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.376556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171e80-bbef-4c8f-be93-9d0a50c577a3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.387697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.389124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.389811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.390095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.390376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.391073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.392494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236b41f-7e2d-4202-ae88-669d03bd326e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.398065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 408/784] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:11.399412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fe8b0-81bf-4ba8-901b-16100a8cc2a5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.399705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fe8b0-81bf-4ba8-901b-16100a8cc2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.400011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fe8b0-81bf-4ba8-901b-16100a8cc2a5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.405055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 409/785] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:11.407903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.408229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.408552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.414890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.415717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.416002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.416280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.426855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:11.428061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:11.428365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.429874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:11.448294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:11.449340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:11.450257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:11.450257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:11.450257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:11.450257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:11.454297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 377/786] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:11.455516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.523945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 410/787] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:11.563407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.569091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4b6d5-e2ae-465a-8c1b-51f6069ab65c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.609966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.610963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.611389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.611799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.613436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:11.613614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.615790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.644217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 378/788] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:11.644217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.678669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.727245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652f378-dc0c-4a36-a77d-dde709a7f139 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.739353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.759356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4905734-1f8c-4020-aa1c-cbaed7cea7e0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:11.768123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.769323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.769825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.770529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.783841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:11.797598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:11.799075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:11.800420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:11.804586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 411/789] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:11.809011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:11.809011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:11.809011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:11.809011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:11.810614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.817034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:11.818756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:11.834940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:11.836043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:11.837433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:11.844885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 379/790] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:11.846418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:11.933643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:11.959092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a38e-ec1c-4a1b-bd50-c0ab2a7bd008 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.036751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.038167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.038859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.039506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.041150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:12.042017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.044592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.077069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 380/791] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:12.079581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.091061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bae02-173a-4659-bb8c-ee3cf7ba99e4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.173257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.174228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.176066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.176066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.176066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:12.176066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.181577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.187159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde21f6-c392-4ad5-8d7a-7accef8e0877 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.197999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 412/792] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:12.202596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.240591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.242605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.243301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.243930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.261371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.266869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.268576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.283589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:12.293130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:12.294827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:12.296589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:12.296589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:12.296589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:12.296589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:12.300269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 381/793] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:12.306202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.409870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578e2b7-e8be-4884-b2ce-b7b3abe30b88 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.446846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.448156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.448629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.449128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.453382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf6947-a3d7-48a9-b06f-8d550b4072a7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.461322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.463141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c57d1675-8028-489f-8428-770dc966e6b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:12.482701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:12.483872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:12.484737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:12.484737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:12.484737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:12.484737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:12.487377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 413/794] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:12.488877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d341-2aca-4b96-ab3a-dc4300a2fe33 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.489373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d341-2aca-4b96-ab3a-dc4300a2fe33 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.489837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d341-2aca-4b96-ab3a-dc4300a2fe33 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.495316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.495316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.495316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.495316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.495651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:12.495932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.496125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 414/795] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:12.497112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68d341-2aca-4b96-ab3a-dc4300a2fe33 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.497338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551e54a-5c65-46d2-8357-5ef41685544c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.510400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 382/796] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:12.511612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.511896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.512169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.530561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:12.531608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:12.583009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.596162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.609640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 383/797] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:12.609640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.630919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68d341-2aca-4b96-ab3a-dc4300a2fe33 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.678438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.679603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.680120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.680580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.681527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:12.682043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.683715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.697090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 415/798] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:12.698203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.720060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f5e5c-cdfb-48ce-8769-53add64d52cb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.759353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.760362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.760646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.761341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.776606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.778391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:12.801576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:12.802530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:12.803410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:12.808985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 384/799] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:12.809926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.818535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8956029-f77c-4c80-8f66-58acd9dd4a5b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.857320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.858344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.858632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.858907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.859615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.861161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.861687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:12.876124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:12.877205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 28 11:52:12.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:12.881125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 416/800] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:12.882215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:12.895081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:12.937439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f671-6c4d-4f22-aa80-5e7f381180d6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:12.973724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:12.974857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:12.975343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:12.975752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:12.984511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:12.986240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:12.991449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5a420-eab6-41a1-b96c-9823bb68965e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:13.003577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:13.004727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:13.005721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:13.008174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 385/801] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:13.009213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.027176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.027851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.028180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.028459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.029507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:13.029818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.031430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.044995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 417/802] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:13.045796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.110383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.116796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bd6a-84ff-46d4-8338-799e99c3a02c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.118353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.147190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827db7c2-b402-4d27-8a2a-b91fac7ff085 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.150466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.151480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.151762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.152036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.152787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:13.153122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.154596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.163659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 386/803] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:13.163659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.190603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.191365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.191715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.192110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.192997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:13.193309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.194751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1a162-7b14-430a-b838-ffb7ad24f4d4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 418/804] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8a94e-a694-4852-b98a-2ba59ec6414e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8a94e-a694-4852-b98a-2ba59ec6414e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8a94e-a694-4852-b98a-2ba59ec6414e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 419/805] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:13.217710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8a94e-a694-4852-b98a-2ba59ec6414e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.272573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3705-aca2-4ff7-8dd9-bfc52a302bb5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.307684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.308572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.308848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.309143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/81b68417e9e043b3abc779986564daff/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.320477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8a94e-a694-4852-b98a-2ba59ec6414e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.324543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=81b68417e9e043b3abc779986564daff, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.326381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.352944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 387/806] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:13 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/81b68417e9e043b3abc779986564daff/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:52:13.352944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.363035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.363978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.377714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.378459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.378910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.379193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.380402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:13.380717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.382255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ddb36-debd-4582-9ad8-4df61cb4e64a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.390459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 420/807] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:13.391839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.392126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.392403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.416307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:13.418263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:13.480883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e6fc-88b5-4d74-abfe-4fa4491178db None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.509823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 421/808] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:13.510965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.521271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.522419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.522948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.523410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.537890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.539765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:13.565072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:13.566004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:13.566949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:13.566949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:13.566949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:13.566949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:13.567594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 388/809] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:13.568205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.629148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048d272-8910-40f9-8b89-925c4fc64b24 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.650143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.657147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:13.679570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.680249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.680529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.680803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.682114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:13.683778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:13.695507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7650-4841-45f5-8671-aaab29565137 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.715181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 422/810] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:13.716390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.745512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:13.746891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:13.747509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:13.748137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:13.762732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:13.771687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:13.772698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:13.773605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:13.777171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 389/811] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:13.786545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:13.957647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81957aa-22d4-460f-9761-859e4a66a341 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.970875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff6717-abe6-4aeb-8323-59baf2616ceb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:13.990046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.003963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955d51b-d182-4dff-a001-017934646395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.004760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d51b-d182-4dff-a001-017934646395 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.005135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d51b-d182-4dff-a001-017934646395 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.005432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955d51b-d182-4dff-a001-017934646395 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.006197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d51b-d182-4dff-a001-017934646395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:14.006508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d51b-d182-4dff-a001-017934646395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.008122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955d51b-d182-4dff-a001-017934646395 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.017363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.018266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.018691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.018963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.019308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 390/812] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:14.019695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:14.020040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.020713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136825f9-25ef-45cf-ae2b-9d1d94ab226a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.021007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136825f9-25ef-45cf-ae2b-9d1d94ab226a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.021287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136825f9-25ef-45cf-ae2b-9d1d94ab226a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.021502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7dce-6aa0-4859-9825-15f7d160de55 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.022796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 391/813] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:14.027095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136825f9-25ef-45cf-ae2b-9d1d94ab226a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.039196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 423/814] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:14.040304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.041080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.041359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.061594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:14.063684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:14.095218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.140554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 424/815] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:14.141807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.170861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136825f9-25ef-45cf-ae2b-9d1d94ab226a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.219543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.221019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.221301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.221572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.237302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.239201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.261839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056b72e-6c1e-4110-b539-e10b32a72477 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:14.262858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:14.263785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:14.264699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:14.264699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:14.264699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:14.264699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:14.265866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 392/816] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:14.266476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.310223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.311523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.311796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.312062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.313192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.323060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.324516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:14.339281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:14.340202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:14.341096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:14.345594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 425/817] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:14.346391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.376325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.399643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade76b0-3d15-4f97-9fbb-1e11babb3cdb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.449890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.450703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.451008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.451375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] PATH_INFO: `/v3/services/network` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.452020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.453546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.459996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb4951-fd82-4e8c-b0ee-5fd5aea9c30f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:14.465777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:14.466879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:14.468273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 393/818] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:14.469080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.510799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.511857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.512338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.513297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.515259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:14.515767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.518035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.540445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 426/819] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:14.540445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.593509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d7b4-41fa-4d24-b1f1-dfb6eb71b827 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.595568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.631386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.634724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.635617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.636085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.636545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.637569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:14.638069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.641244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.646594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94f0d-a79a-4aba-a944-bbe8e8762950 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.655822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 394/820] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:14.657358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.688412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.689166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.689494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.689769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.690493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] RBAC: Adding query filter params (name=network) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:14.690794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.692215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.697655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 427/821] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:14.698259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.767934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb9769-75b6-4737-b783-ca7ccf27f041 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.800206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941aa69-f7ef-4574-a4be-2217fcbbf394 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:14.811890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.812969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.813481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.814026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.821785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.823542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.834767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:14.835667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:14.835945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:14.836216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/ee9dc1ba0084428c9da313ebda0458e0/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:14.838392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=ee9dc1ba0084428c9da313ebda0458e0, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:14.839818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:14.841365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:14.842798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:14.843767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:14.845775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 395/822] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:14.846676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.865836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 428/823] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:14 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/ee9dc1ba0084428c9da313ebda0458e0/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:52:14.866884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:14.891943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:14.893370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.056081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b63-9062-48d4-9c2b-0970edb67196 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.078250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6602c7-9431-4c42-8d26-ab44c1a3d779 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.126478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.126690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.127068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.127174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.127331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.127434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.127601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.127695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.128311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] RBAC: Adding query filter params (type=network) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:15.128479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:15.128592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:15.128739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:15.130263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:15.130805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:15.136624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 396/824] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:15.143999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.148696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 429/825] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:15.158204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.266333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.268339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.296573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3e1fc-bcf1-46a1-9ee2-011082be9c58 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.332299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a56b26-9277-46b0-87d9-665b1415db98 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.356238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.356926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.357247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.357787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.358620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:15.360021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:15.399411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.399600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 397/826] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:15.400053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.400328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.400596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.497171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:15.498820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc8d27-964b-4f59-b8f4-a00903857c86 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:15.587054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.626171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 430/827] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:15.633002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.634761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a915-7ce0-4a38-b63f-6bd5c26bbc11 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.635204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a915-7ce0-4a38-b63f-6bd5c26bbc11 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.635644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a915-7ce0-4a38-b63f-6bd5c26bbc11 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.637458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 431/828] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:15.642227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.642763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.643235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.661426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:15.663017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:15.748990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 432/829] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:15.757704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc9c74-7b40-4b4b-9187-a6b99429bee6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.774668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.802140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.802961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.803240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.803511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.818635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:15.820744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:15.823670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:15.845977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:15.846950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:15.848091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:15.848091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:15.848091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:15.848091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:15.848091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 398/830] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:15.852855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.902889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:15.911366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713ad6-5dd7-40e4-a70b-0a2d509d1800 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:15.948569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:15.949429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:15.949777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:15.950114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:15.961570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:15.973813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:15.974809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:15.975740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 433/831] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:52:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:15.984932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:15.985091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c445d-22fe-4471-9d6a-b53470e713d0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.018016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.018686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.018963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.019233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.019956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.020259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.021747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.034421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 399/832] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:16.039904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.095367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29614a-df76-4f5d-8665-8674d0e4d0bd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.118719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.139688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.140528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.140926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.141299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.142644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.143050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.144936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.157925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 434/833] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:52:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:16.163119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.174715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53b4d-974a-4b9d-a1cc-493a91df51ec None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.192969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.213841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.215068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.215579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.216041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.224220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.226171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6c770a3d-908d-44ad-a7e4-0c3d1a336a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:16.243568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:16.244507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:16.245445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:16.250797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 400/834] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:16.250797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0343ed-fb35-41a9-be13-3a0e139d9564 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.250797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0343ed-fb35-41a9-be13-3a0e139d9564 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.250797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0343ed-fb35-41a9-be13-3a0e139d9564 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.253834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 401/835] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:16.253834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0343ed-fb35-41a9-be13-3a0e139d9564 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.285778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc65ef-d2e7-4403-8fc8-081fdf4fe297 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.320511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.321344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.321624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.322012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.332800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.334275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4553ebf1-bce9-434f-b58c-2ffa11cb3cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:16.352455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:16.353394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:16.354290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:16.354290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:16.354290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:16.354290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:16.354290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 435/836] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:16.355369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.355648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.355830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.366709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0343ed-fb35-41a9-be13-3a0e139d9564 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.373958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:16.374865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:16.381964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.406888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.407639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.407915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.408185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.409002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.409312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.410986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.425914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 402/837] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:16.431287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.440763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.447165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 436/838] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:16.447924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.549445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d90b92-1cab-41b8-be0c-799bcc9151a2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.553888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06dc93-a5bf-493b-bbfe-8477f3e9fa70 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.587476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.588257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.588545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.588823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.589713 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.590027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.590123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.590794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.591068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.591332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.591489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.592048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.592364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.593864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.605580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 437/839] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:16.605580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.612932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 403/840] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:16.619415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.664225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.685629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.732748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f529-a6e3-4082-8604-22556ce31e23 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.740595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0678b-6d80-45b6-b8a3-f2da025ecbba None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.773901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.774737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.775005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.775268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.775936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.777540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:16.790998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:16.792073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:16.792932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:16.793745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.794166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 438/841] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:16.794239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.794239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.794239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.800752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.809202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.810752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:16.828051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:16.829080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:16.830018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 404/842] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:16.836221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:16.920097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546abfc-7538-44e2-b269-97f8c2341505 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.949934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.960115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:16.961188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:16.961612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:16.962046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:16.962898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:16.963346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:16.964019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:16.965047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:16.975769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbef07-75ee-49eb-9cc8-483ddc2265e7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:16.976116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 439/843] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:16.977282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.017510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.018212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.018498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.018778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.019635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:17.019946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.021433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.051141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 405/844] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:17.052639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.091115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cad10-9d6e-4c3d-9ef3-5304183b721a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.127459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.128139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.128423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.128693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.129433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:17.129746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.131131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583a475-837e-441b-ad63-cdccd8b797ce None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.144932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 440/845] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:17.144932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a274df-0081-4d94-8b92-fe10bef3bf12 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.144932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a274df-0081-4d94-8b92-fe10bef3bf12 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.144932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a274df-0081-4d94-8b92-fe10bef3bf12 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.144932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 441/846] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:17.145460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.145635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.145913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.163780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:17.165053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:17.176776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61411299-bc42-4731-a414-9c24e42d5f37 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.219623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.220469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.220845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.221279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.227312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.246534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.251705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 442/847] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:17.281909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.308831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.310860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.396164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff6565-d6cd-458f-8514-d5c4325c294d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.432845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.433690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.434036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.437005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.437178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 406/848] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:17.441527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.447383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:17.457582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:17.458670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:17.459679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:17.464937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 443/849] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:17.468161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.501972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.521268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.550125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914cf4-27fd-4d65-a312-4d8ee866cc50 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.582847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce9835-2b84-4a73-b990-7db61e329d21 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.622117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.623169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.623590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.624097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.625057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:17.625504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.627155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.637878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 444/850] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:17.648444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.712705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.713595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.713873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.714142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.729460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.731325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:17.751955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:17.752924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:17.754132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:17.754132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:17.754132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:17.754132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:17.754132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 407/851] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:17.758724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.775144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f6f9e-5800-4073-bab6-bf22d661e4ff None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.788468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.826121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.826989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.827265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.827542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.839053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.840432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:17.859866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:17.860868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:17.866844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:17.866844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:17.866844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:17.866844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:17.866844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 445/852] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:17.869499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.894224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafe362-ff64-4d55-8f81-aaaa6ccf1261 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:17.896641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:17.932537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:17.933272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:17.933549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:17.933815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:17.934536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:17.934844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:17.936333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:17.948597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 408/853] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:17.953923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:17.979494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16438942-9f5b-4426-b696-bdb12efd0028 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.016040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.017110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.017617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.018209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.019008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:18.019385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.020842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.040928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 446/854] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:18.040928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.064743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.073044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f9d6-faaf-4b21-aec8-299ebdb6897d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.112820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.113564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.113841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.114112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.120788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.122278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:18.139847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:18.140953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:18.142029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:18.145036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 409/855] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:18.147606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.170741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ea496-6f55-473a-aaa3-0dd5ae70fc67 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.178155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.214324 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.215403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.215693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.215969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.226068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.227531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:18.246787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:18.247840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:18.248948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 447/856] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:18.253819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.271940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202943a-5b62-4eac-85d8-1b41785c88c1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.310861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.311564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.311844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.312113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.312955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:18.313243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.314828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.328967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 410/857] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:18.335198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.350959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.376500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804ef3-c0a2-4c24-8460-dd848ab52ffc None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.412918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.413702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.413989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.414266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.415673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:18.415673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.416979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.430987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.438787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 448/858] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:18.444127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.449168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57562de-bf9b-4969-a226-0370be55ab8f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.484567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.485610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.486109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.486584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.487737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=81b68417e9e043b3abc779986564daff) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:18.488247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.489996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.501239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 411/859] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/role_assignments?user.id=81b68417e9e043b3abc779986564daff&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:18.567214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6a4d3-1fce-4232-b594-d7a66fd28816 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.608607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.609426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.609589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.609857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.613825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.685209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:18.690439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a31b7-3da2-47b1-953a-2955b968cdf6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:18.806917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 449/860] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:18.815172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.854175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30fca0-59bf-4a31-86e5-34127173f170 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.854495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30fca0-59bf-4a31-86e5-34127173f170 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.854729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30fca0-59bf-4a31-86e5-34127173f170 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.856730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 450/861] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:18.863555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.863840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.864109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.881728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:18.882940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:18.916765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:18.940145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc546bfb-5521-446a-be49-1e2151f7f668 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:18.960302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 451/862] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:18.983089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:18.984724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:18.986837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:18.987451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:18.987921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:18.999978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.001793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.010701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:19.020947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:19.021944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:19.022837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:19.022837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:19.022837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:19.022837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:19.030237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 412/863] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:19.031705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.087459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710cde3-051f-4934-a65d-583ab3736b6d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.124387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.125289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.125635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.125837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.126644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.127865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.136199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5499a3a-ee80-401e-9a20-850feba8cbe7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:19.140708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:19.141665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:19.142480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:19.146950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 452/864] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Jun 28 11:52:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:19.147824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.181535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.187736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.188424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.188703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.189019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.189763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:19.190075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.191525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.210849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 413/865] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:19.212482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.263876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf188f-0237-4085-989d-7c5916c526af None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.273331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.298931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.299607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.299887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.300160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.300873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:19.301221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.302595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.312292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 453/866] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:19.313357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.315104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8adc12-8be6-462e-8b1d-c902ef2f45bc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.351313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.352122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.352400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.352670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.360568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.362063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-73ef05d3-8697-4d25-ae4c-4e27b5365326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:19.375909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:19.376812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:19.377756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:19.381062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 414/867] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:19.383025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab533-58d6-49cd-9435-ba34b4910605 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.383309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab533-58d6-49cd-9435-ba34b4910605 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.383580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ab533-58d6-49cd-9435-ba34b4910605 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.385194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 415/868] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:19.385618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ab533-58d6-49cd-9435-ba34b4910605 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.419843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.428022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a3d2f-dd0a-4cfb-9280-ef8e72cf4474 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.464517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.465234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.465518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.465792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.466622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:19.466927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.468884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef53fc8-e195-457c-aefe-a6a06f628d5d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.484950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 454/869] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:19.484950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.484950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.484950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.501899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:19.503476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:19.503930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ab533-58d6-49cd-9435-ba34b4910605 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.527045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.543125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.543813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.544095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.544391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.545644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:19.545644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.547164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.566176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 416/870] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:19.567908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.601070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 455/871] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:19.625850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:19.708672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9aed3-886e-4c46-8aba-b979a37bd0d0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.733042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:19.750511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de450d52-e202-4136-af1c-83ebf58874ec None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:19.757922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.758903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.759804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.760278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.761343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=ee9dc1ba0084428c9da313ebda0458e0) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:19.762145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.764417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83111f7-870b-4797-a00e-fcf3bc96b606 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.775419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 417/872] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:19 2022] GET /identity/v3/role_assignments?user.id=ee9dc1ba0084428c9da313ebda0458e0&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:19.798498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:19.799538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:19.799997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:19.800444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:19.801455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:19.803191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a770a4-5c81-4c71-9f8b-f43cb20926ae None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:19.822337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 456/873] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:19 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:20.734144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149afc8-a7a3-46f6-9dc8-853c684ccb1e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:20.735061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149afc8-a7a3-46f6-9dc8-853c684ccb1e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:20.735501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149afc8-a7a3-46f6-9dc8-853c684ccb1e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:20.737425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 418/874] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:20.746070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:20.746858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:20.747321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:20.764983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:20.766566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:20.840574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 457/875] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:20.864816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149afc8-a7a3-46f6-9dc8-853c684ccb1e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:20.913119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:20.970297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149afc8-a7a3-46f6-9dc8-853c684ccb1e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.011353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.012197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.012482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.012751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.024110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:21.032258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:21.033190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:21.034022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 419/876] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:21.039443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.116653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:21.142191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d0bf-1bcd-4de6-bacf-a36071dc8a9b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.176962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.177643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.177924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.178194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.178938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:21.179251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.180607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.189371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 458/877] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:21.194886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.296706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479733d-f267-4088-b013-734f83b9f38c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.299481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:21.336849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.337681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.337963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.338235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.349377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.350859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:21.369141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:21.370093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:21.370979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:21.370979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:21.370979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:21.370979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:21.370979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 420/878] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:21.377505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.482526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5baf37-0069-45f8-a261-8899911c6ea3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.517908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.518966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.519473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.519952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.520884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:21.521410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.522984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0580636-0dbd-431e-9edb-40000050967e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.525742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:21.537693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 459/879] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:21.543578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.618692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36bf1-0094-47e3-b9e0-a54b4ebf32f4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.619312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36bf1-0094-47e3-b9e0-a54b4ebf32f4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.619804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36bf1-0094-47e3-b9e0-a54b4ebf32f4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.621666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 460/880] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:21.627550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.628115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.628612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.646557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:21.648286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:21.654876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb964-3092-452e-a584-dfed6b0b2b28 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.697560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.698386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.698660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.698924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.708150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.709893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:21.727903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:21.728953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:21.730269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 421/881] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:21.737347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.740952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 461/882] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:21.763045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.764015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:21.860568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14d20e-9ff5-4e28-accb-71c1b6aa6636 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.879636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c0767-1c5e-4ffe-b7c7-07d7c00ab4c0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:21.896280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.897022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.897277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.897536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.898386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:21.898693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.900342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.909500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:21.919706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:21.920806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.921316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.921803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.922732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:21.924364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bede5-9127-4bc7-aee3-79c96ea2086c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:21.925152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 422/883] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:21.927140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:21.942616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 462/884] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:21 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:21.945296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3279c-08d9-4b4b-afea-caf7a92bcee8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.945809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3279c-08d9-4b4b-afea-caf7a92bcee8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.946534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3279c-08d9-4b4b-afea-caf7a92bcee8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.948938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 463/885] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:21.954628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ece-d5f8-4ab1-b77d-726f6ca2d595 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:21.955868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ece-d5f8-4ab1-b77d-726f6ca2d595 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:21.956625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ece-d5f8-4ab1-b77d-726f6ca2d595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:21.982565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-321f9ece-d5f8-4ab1-b77d-726f6ca2d595 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:21.984878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-321f9ece-d5f8-4ab1-b77d-726f6ca2d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:22.017553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.048172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fea93-6fdc-4ff5-887c-ae2a485ad96b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.066627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 464/886] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:22.068384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aac2d-f6c3-4f7e-b6ac-777538e07db4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.068989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aac2d-f6c3-4f7e-b6ac-777538e07db4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.069564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aac2d-f6c3-4f7e-b6ac-777538e07db4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.071218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 465/887] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:22.077236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.077848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.078338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.095985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:22.097460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:22.098882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.099734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.100023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.101202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.177854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 466/888] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:22.178625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.182940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.184545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.234427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.266881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.295727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 423/889] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:21 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 28 11:52:22.295727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.304639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be8417-477c-42f6-a74c-8d5a4ea74e04 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.341289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.342191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.342710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.342710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.343394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.344783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:22.358197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:22.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:22.360016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 467/890] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:22.360802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.406862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887134-6b0a-41e0-b4fb-6042132bb160 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.442407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.443151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.443422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.443695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.444378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.445782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:22.457812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:22.458773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:22.459601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 424/891] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:22.460374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.474624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d22190-9aba-4701-8752-e5dae595d91f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.483687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.506981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.513840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.515023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.515448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.515727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.526740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.528210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:22.548412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:22.549395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:22.550302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:22.550302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:22.550302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:22.550302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:22.550302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 468/892] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:22.550575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.573733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800a3-6b76-4e7a-a313-ec80484e9ff6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.608538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.609240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.609530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.609802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.610507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:22.610812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.612184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.617268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 425/893] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:22.622442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.658928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88426639-5d1f-4cee-80bc-b6e3db985dcd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.695833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.696573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.697088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.697336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.698120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:22.698495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.699948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.705023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 469/894] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:22.709837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.723457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5eb0c-6817-420a-a7e8-111586437854 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.725151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.737118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.764281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.765254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.765537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.765812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.766538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:22.766845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.768228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.781010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 426/895] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:22.786066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.829254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987bdaf-409f-4684-8b37-741b94ccea98 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.868225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.868938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.869514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.869793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.870657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:22.870967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.872351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.877665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 470/896] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:22.878347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.898400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27988c09-77c9-43e2-8a4c-b00b12d609f0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:22.937004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:22.937693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:22.937987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:22.938225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:22.938914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:22.939222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:22.940629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:22.945210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 427/897] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:22.946042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:22.986366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:22.992739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:23.007812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e4429-a177-4615-a54a-9352ce672e23 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:23.071717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:23.073835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:23.074305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:23.075233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:23.086380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:23.088233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:23.108755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ae50f-e8b3-42ce-abba-e4f2dfb13471 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:23.109842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:23.110940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:23.112115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:23.115501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 471/898] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:23.121411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:23.153167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:23.154206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:23.155423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:23.155759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:23.156805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] RBAC: Adding query filter params (interface=public, service_id=ecffa9b275df418f828b9cf7387e11e2, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:23.157184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:23.158582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:23.164493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 428/899] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:22 2022] GET /identity/v3/endpoints?service_id=ecffa9b275df418f828b9cf7387e11e2&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:23.170156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:23.245020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48a96d-8d3d-41c9-8714-a4df4056150c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:23.265316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:23.278160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90866-14b7-4291-a801-f0b3b2570a89 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:23.285156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:23.285827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:23.286105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:23.286377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:23.287163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:23.287477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:23.288926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:23.306299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 472/900] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:23.307421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:23.316971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:23.318076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:23.319330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:23.319826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:23.320286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:23.321310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:23.322967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbdfdd-a856-473f-934b-716c41e1b54f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:23.330973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 429/901] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:23 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:52:23.422131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6640df8-9ea4-4c2b-90e8-63556be6a510 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:23.464037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:23.464743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:23.465046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:23.465328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:23.466192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] RBAC: Adding query filter params (role.id=ceee4461d8df498da7cdc5d10c0c9619, scope.project.id=5709e926be584e859153090dd261a990, user.id=81b68417e9e043b3abc779986564daff) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:23.467010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:23.468657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cce8bd-19a8-4f1f-b14b-632ea6620136 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:23.475609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 473/902] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:23 2022] GET /identity/v3/role_assignments?user.id=81b68417e9e043b3abc779986564daff&scope.project.id=5709e926be584e859153090dd261a990&role.id=ceee4461d8df498da7cdc5d10c0c9619 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:24.032209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93836fb-85c7-4ff8-a5f8-d0aa5f2e0027 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.032745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93836fb-85c7-4ff8-a5f8-d0aa5f2e0027 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.032745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93836fb-85c7-4ff8-a5f8-d0aa5f2e0027 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.034197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 430/903] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:24.039661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.040477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.041007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.058358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:24.059922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:24.125627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:24.130408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 474/904] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:24.156555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93836fb-85c7-4ff8-a5f8-d0aa5f2e0027 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:24.272467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93836fb-85c7-4ff8-a5f8-d0aa5f2e0027 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:24.306651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:24.307386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.307660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.307928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.308569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:24.309974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:24.322232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:24.323235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:24.324116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 431/905] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Jun 28 11:52:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:24.331739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:24.434442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5236f1-0006-46fa-b108-19e921cc6719 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:24.469094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:24.469773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.470064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.470337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.471061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:24.471360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:24.472742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:24.478017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 475/906] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:24.484082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:24.504924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:24.597265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c16995-236d-4d67-ac74-8a6b081f47d8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:24.632668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:24.633618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.634050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.634463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.635320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:24.635771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:24.637310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:24.642952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 432/907] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:24.651322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:24.714103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:24.753554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aaf8b-e661-4913-93c6-d37094c80f37 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:24.789220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:24.790026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.790597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.790957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.791907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] RBAC: Adding query filter params (interface=public, service_id=7bcc2733108747698b5575c4851a869b, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:24.792309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:24.793840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa7390-995c-4141-8009-22e53ac5c7bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:24.800477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 476/908] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:24 2022] GET /identity/v3/endpoints?service_id=7bcc2733108747698b5575c4851a869b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:24.805847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:24.893676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:24.914167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796fd10-0cd8-4c3a-9cf9-ac22c4483780 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:24.949125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:24.950180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:24.950679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:24.951170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:24.952122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:24.953724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c81b4e-7c61-43d5-be03-82189c1865a7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:24.960342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 433/909] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:24 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:52:25.212142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6f17-1c1c-4a1d-a24a-4b06d8072ef3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.213124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6f17-1c1c-4a1d-a24a-4b06d8072ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.213622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6f17-1c1c-4a1d-a24a-4b06d8072ef3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.215445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 477/910] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:25.221240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6fe8-5832-4ffd-ae68-29a5ba1bec58 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.222076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6fe8-5832-4ffd-ae68-29a5ba1bec58 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.222552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba6fe8-5832-4ffd-ae68-29a5ba1bec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.241607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-08ba6fe8-5832-4ffd-ae68-29a5ba1bec58 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:25.243176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-08ba6fe8-5832-4ffd-ae68-29a5ba1bec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:25.313272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 434/911] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:25.336578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be6f17-1c1c-4a1d-a24a-4b06d8072ef3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:25.450565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be6f17-1c1c-4a1d-a24a-4b06d8072ef3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:25.479371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:25.488035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:25.488709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.489066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.489559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.490518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:25.492090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a711b-20d6-4514-a3bc-0a05c25747d1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:25.507112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251d6eb-d644-4d26-98ab-9fbb678cca74 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.507756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251d6eb-d644-4d26-98ab-9fbb678cca74 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.508224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251d6eb-d644-4d26-98ab-9fbb678cca74 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.509834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 478/912] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:25 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:25.510097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 435/913] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:25.514568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522078d-19d4-4af2-857e-fde21b719273 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.514568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522078d-19d4-4af2-857e-fde21b719273 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.514568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522078d-19d4-4af2-857e-fde21b719273 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.514568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 479/914] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:25.516667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.516978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.517254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.520706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.521163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.521490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.535602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:25.536846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:25.540378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:25.541619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:25.626934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 480/915] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:25.634896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 436/916] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:25.654361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:25.660766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:25.663561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:25.779075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f8df-0dff-4224-bfe9-1992184b52fd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:25.797795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:25.798290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbff901-93f6-42ad-86a0-c5d533ada888 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:25.827413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:25.828873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.829592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.830444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.831572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:25.834061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:25.847057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2610-f697-4522-961b-0cff63657446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:25.848170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2610-f697-4522-961b-0cff63657446 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:25.848657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2610-f697-4522-961b-0cff63657446 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:25.849160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2610-f697-4522-961b-0cff63657446 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:25.849912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:25.850894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:25.851750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 481/917] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:25.858217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:25.864996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c2610-f697-4522-961b-0cff63657446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-932c2610-f697-4522-961b-0cff63657446 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:25.875139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:25.876086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:25.877399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:25.879834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 437/918] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:25.888463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2610-f697-4522-961b-0cff63657446 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:25.960207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:25.967705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030e49e-5990-445f-9cd6-47a9e4f62c18 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.004822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2610-f697-4522-961b-0cff63657446 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.005505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.006186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.006465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.006739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.007549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:26.007861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.009438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.014991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 482/919] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:26.020220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.043387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.044456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.045085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.045657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.046694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:26.047284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.048931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.058596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 438/920] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:26.066078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.075383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:26.154159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb28652-7bc7-4d23-92ce-b8ea943e4f59 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.191922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab704bf-e1d1-4da7-ac19-9777260c7601 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.199874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.201398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.201915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.202400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.203785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:26.204708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.206630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.213877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 483/921] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:26.224223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.224361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:26.235659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.236770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.237085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.237508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.251475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.253426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:26.275557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:26.276492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:26.277547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:26.277547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:26.277547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:26.277547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:26.281019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 439/922] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:26.282845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.355668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e80c-c6e6-474e-8a89-2ea97715b94b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.363866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:26.395347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.396023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.396308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.396582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.397359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.398889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.410546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14e94-8333-4103-8ec9-ac3b2f8c8d82 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.423433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 484/923] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:26.459084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.459778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.460059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.460436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.461293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:26.461608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.463055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.478365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 440/924] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:26.484882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.603996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc559284-06e8-45b4-8acd-f042353705d1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.641165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.641996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.642271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.642547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.650380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.651827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:26.667451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:26.668449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:26.669539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 485/925] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:26.676629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.712381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:26.800331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac583407-b5da-4dbe-ace9-90b97e26f319 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:26.837049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:26.838035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:26.838482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:26.838943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:26.839958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:26.840430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:26.842847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:26.862338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 441/926] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:26.869036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:26.894875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:26.973516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74c01-128d-49ec-8898-df97bfca763e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.007641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.008695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.009145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.009582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.021148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.022851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-e3106609-454f-4053-b94f-15c6104fc9eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:27.040040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:27.041165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:27.042150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:27.042150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:27.042150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:27.042150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:27.048134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 486/927] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:27.049439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.100023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329d37-6b47-454c-a12e-02eb6c7b5fe0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.100297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329d37-6b47-454c-a12e-02eb6c7b5fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.100572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329d37-6b47-454c-a12e-02eb6c7b5fe0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.102331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 487/928] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:27.108931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.109297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.109568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.127213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:27.128319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:27.149081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.159851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c245e-83ec-442f-8783-7b3534f0f473 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.196450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.197483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.197954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.198495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.199385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:27.199864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.201578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.204936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 488/929] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:27.215306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 442/930] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:27.223633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.228373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.230550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.357303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9926dae-8ee8-40c9-a10a-1b2b9a22f09f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.358644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76359685-67cf-4dd2-b772-08fc08c8a6c6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.404573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.405270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.405571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.405825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.406060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.406163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.406338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.408937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.410670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.410670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.415355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.416797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:27.422236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:27.424040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 28 11:52:27.426030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:27.436957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 443/931] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:27.436957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:27.437197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:27.438092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:27.438981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 489/932] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:27.440810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.455347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.518422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.562905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c76490-119d-4fec-b4e6-ddb1ac42c06c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.567082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a558e70-13e2-45c5-a27a-4857c96553ab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.599828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.600650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.601147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.601521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.602436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:27.602845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.604509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.613054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 444/933] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:27.616841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.617833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.618498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.618774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.619039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.619826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:27.620128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.621604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.639840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 490/934] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:27.646121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.726821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.755537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9f707-114c-4c70-869d-43901bc909eb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.786493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571b8af-4fd3-429b-b78d-25bf44af5321 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.803817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.804940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.805199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.805492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.806199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:27.806500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.808365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864cd22-f13f-4011-9884-abb9e302ac1e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.811463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:27.820828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 445/935] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:27.820828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56731e67-cf34-4e41-bec1-c49f5e77c532 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.820828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56731e67-cf34-4e41-bec1-c49f5e77c532 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.820828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56731e67-cf34-4e41-bec1-c49f5e77c532 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.820828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 446/936] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:27.825289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56731e67-cf34-4e41-bec1-c49f5e77c532 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:27.828564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.829532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.829817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.830090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/81b68417e9e043b3abc779986564daff/roles/ceee4461d8df498da7cdc5d10c0c9619` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.832345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=81b68417e9e043b3abc779986564daff, role_id=ceee4461d8df498da7cdc5d10c0c9619)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.833824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8c81d-3938-4a29-baae-f62c7ea0ddc5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:27.857030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 491/937] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:27 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/81b68417e9e043b3abc779986564daff/roles/ceee4461d8df498da7cdc5d10c0c9619 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:52:27.857030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.857030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.857030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.880200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:27.881800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:27.942332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56731e67-cf34-4e41-bec1-c49f5e77c532 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:27.981148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 492/938] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:27.987108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:27.987808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:27.988086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:27.988354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:27.989151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:27.990564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.008867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.011362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:28.025090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 447/939] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:28.143095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9788ee-4869-49db-b29d-05d8c66e2fe6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.159013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:28.189588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.190423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.190703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.190975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.191631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.193114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:28.206930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:28.208009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:28.208853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 493/940] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:28.215093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.322750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b85764-33c7-4329-b00e-224372e02f31 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.359191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.360277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.360758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.361272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.362194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:28.362697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.364298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.369957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 448/941] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:28.375659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.482423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e206a3-3c87-41ba-a102-cde682ee4b7f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.516571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.517286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.517565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.517835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.518553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:28.518855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.520241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e8540-d4b9-4216-a3d0-6444cd92e370 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.525971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 494/942] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:28.531677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.600184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:28.641483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7a6fe-b51f-44ce-9cdb-2c51695c8286 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.668047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2a7b0-837d-468d-960d-df51e26d7454 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.668274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2a7b0-837d-468d-960d-df51e26d7454 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.668480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2a7b0-837d-468d-960d-df51e26d7454 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.670159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 495/943] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:28.675497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.675790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.676064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.677373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.678251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.678715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.679173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.680167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] RBAC: Adding query filter params (interface=public, service_id=9aa3fac7ac3d44c3b1f1931414899e27, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:28.680664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.682512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.688541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 449/944] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:28 2022] GET /identity/v3/endpoints?service_id=9aa3fac7ac3d44c3b1f1931414899e27&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:28.694044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.694552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:28.695949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:28.730533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:28.788953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 496/945] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 28 11:52:28.813979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:28.816407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5f9b3-361f-4a2c-9a66-b2fcead08565 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.842663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:28.852708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.853263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.853540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.853809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.854532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.856335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.862989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 450/946] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:28 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 11:52:28.935821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b232ae-c69f-4e2d-a9e1-6adf8c2e702d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:28.974418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:28.975534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:28.976038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:28.976529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:28.977481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:28.979203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:28.981302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING keystone.server.flask.common [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 28 11:52:29.002051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.sql.core [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 28 11:52:29.002051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 28 11:52:29.002051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [parameters: {'id': '4bf227b6695c4191a9936323431b2cfc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 28 11:52:29.002051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bf227b6695c4191a9936323431b2cfc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bf227b6695c4191a9936323431b2cfc. Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:52:29.020235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result.read() Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 28 11:52:29.021197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application yield resource Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.session.rollback() Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.force_reraise() Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise self.value Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application session.commit() Jun 28 11:52:29.022172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._prepare_impl() Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.session.flush() Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._flush(objects) Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application compat.raise_( Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application flush_context.execute() Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 28 11:52:29.023065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rec.execute(self) Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 28 11:52:29.023998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result.read() Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:52:29.024811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [parameters: {'id': '4bf227b6695c4191a9936323431b2cfc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:29.027024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:29.027993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bf227b6695c4191a9936323431b2cfc. Jun 28 11:52:29.029237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:29.031473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 497/947] 10.222.0.55 () {60 vars in 1140 bytes} [Tue Jun 28 11:52:28 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:29.038960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:29.150098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c100a-e40b-45ed-9170-abedb7802690 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:29.192938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:29.192938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:29.192938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:29.192938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:29.202122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:29.203619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-308eed20-e8fd-485e-a25e-a8157a0f34eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:29.224103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:29.225072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:29.226489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 451/948] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:29.238126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:29.269768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:29.351636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42df03f-a58e-47ca-b952-251a870ac6a4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:29.386441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:29.387095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:29.387386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:29.390292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:29.390292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:29.390292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:29.394680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a1b21-a73a-4942-bc9f-592c1679f91f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:29.423767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 498/949] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:30.152421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71956f-eb76-4135-9b20-a742f4819d5a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.152961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71956f-eb76-4135-9b20-a742f4819d5a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.152961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71956f-eb76-4135-9b20-a742f4819d5a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.154685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 452/950] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:30.160191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3cab9-fd34-49e0-8312-3f4388e34b9a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.160475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3cab9-fd34-49e0-8312-3f4388e34b9a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.160747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3cab9-fd34-49e0-8312-3f4388e34b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.178355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-94e3cab9-fd34-49e0-8312-3f4388e34b9a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:30.179861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-94e3cab9-fd34-49e0-8312-3f4388e34b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:30.247411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 499/951] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:30.258146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4993-6e88-4c10-b128-d79f5b1f6f39 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.258432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4993-6e88-4c10-b128-d79f5b1f6f39 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.258707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee4993-6e88-4c10-b128-d79f5b1f6f39 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.260333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 453/952] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:30.265692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.266015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.266323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.272506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee4993-6e88-4c10-b128-d79f5b1f6f39 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.283540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:30.285000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:30.295193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.334880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.361455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 500/953] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:30.383940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.394370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee4993-6e88-4c10-b128-d79f5b1f6f39 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:30.434574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:30.435728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.436148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.436557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.447390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:30.455929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:30.456862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:30.458091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:30.460644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 454/954] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:30.466735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.505332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784ab9-83c2-4b5b-8790-06695ee6c58e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:30.558565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:30.559876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.560420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.560877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.561649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:30.563241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:30.576345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec64b69-df09-4192-a451-d7d9a7d9ef1d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:30.581096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:30.582420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:30.583557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:30.589455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 501/955] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:30.589714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.593906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.606553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.627142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:30.627824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.628104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.628407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.629645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:30.629958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:30.631355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:30.646266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 455/956] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:30.647169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.728209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7b80-e6d9-401f-99e9-6a750558224e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:30.754049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6e032-099c-471c-aec4-6f3fa271df7d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:30.785886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:30.786909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.787643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.788135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.789701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:30.790333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:30.792025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:30.794562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:30.795423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:30.795666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:30.795948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:30.800121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 502/957] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:30.805403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.813298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:30.814786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:30.832677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:30.833740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:30.834618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:30.834618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:30.834618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:30.834618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:30.838725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 456/958] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:30.839634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:30.879056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.928256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:30.985379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e9a07-e120-4621-87a8-e0d8560a34bf None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.026644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36edb79-d6da-4b41-9fc2-0a61912057cf None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.049165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.050585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.051320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.051963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.053439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:31.054182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.057054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99f91f-49b1-4617-be31-c6d47c46577b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.065573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 503/959] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:31.067918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b9d-0f69-4176-b64b-3d02ff8a45db None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.068580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b9d-0f69-4176-b64b-3d02ff8a45db None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.069277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b9d-0f69-4176-b64b-3d02ff8a45db None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.072010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 504/960] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:31.080938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.080938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.080938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.082010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.083148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.083609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.084072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.085291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:31.085794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.088038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.097033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:31.098250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:31.114911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 457/961] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:31.116419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.193543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 505/962] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:31.222286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.245251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.264245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d1db2-3514-4b8d-87c4-19ad3b492dc5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.299269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.311409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.312239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.312517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.312787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.323494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.325289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.340185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9331-959e-4018-bb54-cb4c56c6ea52 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:31.346603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:31.347581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:31.349264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 458/963] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:31.356075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.382222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.383600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.384138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.384690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.385561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.387355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:31.400825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:31.401901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:31.402743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:31.406359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 506/964] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:31.422086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.477753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54585f-d951-4fe9-9e0a-e40efe559dc4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.525957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.527048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.527782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.528718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.530108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:31.530635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.532886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.551259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4040de-e70c-4280-8a4f-66c26dfb5a72 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.557152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 459/965] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:31.565893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.568736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.598798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.609657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.610678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.611945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.612429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.613390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:31.614402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.616659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.623795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 507/966] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:31.632457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.685641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba6795-c7b0-4371-8188-47a2f3a760d5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.721521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.722327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.723397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.723674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.764257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e0c5-8ac9-4a05-9b61-321f93a86cc0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.800941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.802455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f82d-48ca-4879-a3a3-16f8229cb81c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.804288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.813673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.814637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.814937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.815349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.816200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:31.816510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.818522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cb046-4b0d-4179-a694-1641fa66ece4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:31.828952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 508/967] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:31.828952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cc08-52cb-4778-bd94-855d234792a5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.828952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cc08-52cb-4778-bd94-855d234792a5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.828952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cc08-52cb-4778-bd94-855d234792a5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.830116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 509/968] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:31.834172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48cc08-52cb-4778-bd94-855d234792a5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:31.902314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:31.911002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 460/969] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:31 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:31.912510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.912949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.913242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.929940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:31.931161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:31.955277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48cc08-52cb-4778-bd94-855d234792a5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:31.991254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:31.991929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:31.992352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:31.992636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:31.993989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:31.995551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.000349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 461/970] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:32.001217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.028829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 510/971] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:32.030547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.045252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.131380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367075a-d14f-41fd-8681-1ddf26b98a4f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.178477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd859ff-672b-47a7-a62c-7c6e960ca1d6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.179328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.180342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.180767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.181226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.193342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.193871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.195088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:32.219061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:32.220006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:32.220885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:32.220885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:32.220885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:32.220885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:32.221148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.221148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.221148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.221148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.221408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.222430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 462/972] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:32.224202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.228178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.240064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.sql.core [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 28 11:52:32.240064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 28 11:52:32.240064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [parameters: {'user_id': '1bbe064e21614dcf97b9a2ade015eec5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 28 11:52:32.240064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80100) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result.read() Jun 28 11:52:32.248561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:52:32.249510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application yield resource Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application next(self.gen) Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.session.rollback() Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.force_reraise() Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise self.value Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application session.commit() Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 28 11:52:32.250975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._prepare_impl() Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.session.flush() Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._flush(objects) Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application compat.raise_( Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application flush_context.execute() Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rec.execute(self) Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 28 11:52:32.251876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 28 11:52:32.252779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result = self._query(query) Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application conn.query(q) Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application result.read() Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 28 11:52:32.253685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [parameters: {'user_id': '1bbe064e21614dcf97b9a2ade015eec5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:32.254684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 28 11:52:32.255475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.256407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 511/973] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Jun 28 11:52:32 2022] POST /identity/v3/users => generated 160 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:32.260682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.354926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.375061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06c9bd-ec52-43bc-90d5-a48c456e94a8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.393176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe50e9-1b04-412c-bcea-d2e79fc19f2a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.412819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.413522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.413798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.414385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.414998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:32.415308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.416685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.428930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 463/974] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:32.434013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.440373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.441430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.441857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.442273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.453927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.456179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:32.476231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:32.477512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 28 11:52:32.478693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:32.481968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 512/975] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:32.491511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.529606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.555773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724749a0-19b8-47d4-a252-6db6bff0fb36 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.600667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.602559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.603052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.603543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.604192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315468b-9831-4633-bc99-a5b6aaa9a1c2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.613589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.616183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:32.634425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:32.635404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:32.636314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:32.640514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 464/976] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:32.641650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.646103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.649304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.650234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.650695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.651171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.652672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:32.653235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.656395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.717378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 513/977] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:32.755901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634b403-f4b7-4588-8ff4-53408f471fb1 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.794145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.794992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.795272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.795695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.796519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:32.796828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.798337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aabc7c-17ad-4abe-930b-6eba0123d4f2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.812494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 465/978] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:32.818980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:32.919649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:32.930668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16efdb-8b5d-4f87-9808-0b33322c703e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:32.971702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:32.972716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:32.973019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:32.973294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:32.974137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] RBAC: Adding query filter params (role.id=ceee4461d8df498da7cdc5d10c0c9619, scope.project.id=5709e926be584e859153090dd261a990, user.id=81b68417e9e043b3abc779986564daff) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:32.974449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:32.975892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4951f0-fd80-48cb-9305-e612ed810baf None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:32.982769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 514/979] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:32 2022] GET /identity/v3/role_assignments?user.id=81b68417e9e043b3abc779986564daff&scope.project.id=5709e926be584e859153090dd261a990&role.id=ceee4461d8df498da7cdc5d10c0c9619 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:33.253086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e465c-fb1b-4074-83ec-ee6d8b0980e6 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:33.253275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e465c-fb1b-4074-83ec-ee6d8b0980e6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:33.253568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e465c-fb1b-4074-83ec-ee6d8b0980e6 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:33.254936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 466/980] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:33.260776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf504e9-a32d-4cde-9070-0e9744abd079 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:33.261067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf504e9-a32d-4cde-9070-0e9744abd079 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:33.261377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf504e9-a32d-4cde-9070-0e9744abd079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:33.278867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bdf504e9-a32d-4cde-9070-0e9744abd079 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:33.280428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bdf504e9-a32d-4cde-9070-0e9744abd079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:33.351261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 515/981] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:33.375923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e465c-fb1b-4074-83ec-ee6d8b0980e6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:33.448625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:33.494026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e465c-fb1b-4074-83ec-ee6d8b0980e6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:33.532004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:33.532688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:33.532993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:33.533266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:33.534005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:33.535418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43df8fb-2780-4b4d-a0d7-57efa848a8a4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:33.560932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 467/982] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:33 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:34.165055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d547-87f5-4639-ae82-3dafff26a580 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.165595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d547-87f5-4639-ae82-3dafff26a580 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.165595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313d547-87f5-4639-ae82-3dafff26a580 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.167027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 516/983] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:34.172782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.173081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.173380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.192400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:34.193691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:34.281190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 468/984] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:34.307917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313d547-87f5-4639-ae82-3dafff26a580 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:34.400852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:34.421139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313d547-87f5-4639-ae82-3dafff26a580 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:34.460583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:34.461434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.461708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.461977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.462631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:34.464001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:34.476802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:34.477936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:34.478812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:34.481276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 517/985] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:34.484766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:34.540119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:34.599494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2176-57da-4316-9349-5f646f6a2a64 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:34.641074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:34.641744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.642020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.642282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.642986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:34.643280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:34.644645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:34.649811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 469/986] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:34.655713 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:34.755540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf27c8-fc0f-44c3-b57a-4b2474bf4c7c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:34.798551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:34.799234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.799496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.799759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.800466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:34.800769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:34.802165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:34.807621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 518/987] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:34.813610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:34.926313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539829-95d0-48fa-84ac-3008b7080593 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:34.942623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:34.968040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:34.968711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:34.969019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:34.969296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:34.970098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] RBAC: Adding query filter params (interface=admin, service_id=9aa3fac7ac3d44c3b1f1931414899e27, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:34.970410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:34.971791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e70c-ffa7-4e39-ad17-ed9bbe0facc1 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:34.982069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 470/988] 10.222.0.55 () {56 vars in 1249 bytes} [Tue Jun 28 11:52:34 2022] GET /identity/v3/endpoints?service_id=9aa3fac7ac3d44c3b1f1931414899e27&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:34.983706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:34.999599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:35.084894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615abd1-c9cf-41f4-955e-692675de7a92 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:35.119093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:35.119763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.120013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.120284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.121024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:35.122393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbc70b-fd23-48ff-83f0-64f5395c69b9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:35.129147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 519/989] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:34 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:52:35.139637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530b2e1-5b99-4faf-8f50-2938bb258251 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.139954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530b2e1-5b99-4faf-8f50-2938bb258251 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.140358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530b2e1-5b99-4faf-8f50-2938bb258251 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.141955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 471/990] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:35.148046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.148389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.148585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.168152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:35.169603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:35.248512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 520/991] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:35.271048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530b2e1-5b99-4faf-8f50-2938bb258251 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:35.386753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530b2e1-5b99-4faf-8f50-2938bb258251 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:35.422830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:35.423693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.423948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.424217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.424866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:35.426309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:35.438173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:35.439240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:35.440080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 472/992] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Jun 28 11:52:35 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:35.446876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:35.455254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:35.551865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc60cce-df54-4bd7-a679-c48a822b1f57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:35.586442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:35.587118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.587374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.587659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.588369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] RBAC: Adding query filter params (name=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:35.588675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:35.590078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e48bf-f110-44c3-9685-ceae6a8e6b79 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:35.595482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 521/993] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:35.601186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:35.648021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:35.705873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da235d3-193a-41d8-bcad-32ac390c0df3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:35.728808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09987-ff24-4af2-9ccb-2ac0f6913344 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.729134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09987-ff24-4af2-9ccb-2ac0f6913344 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.729431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09987-ff24-4af2-9ccb-2ac0f6913344 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.731320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 522/994] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:35.736090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.736371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.736657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.740479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:35.741184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:35.741453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:35.741727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:35.742433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:35.742739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:35.744110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:35.753113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 473/995] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:35.754732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:35.756203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:35.837491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:35.844637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 523/996] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:35.868363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:35.973460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d00c6-2e52-4e40-afe2-0ecbd37a95d4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:35.980521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:36.008606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:36.009447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:36.009727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:36.009974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:36.010656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:36.012033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:36.023994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:36.025029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:36.026269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 474/997] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:36.031626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:36.135383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd903-abd6-443e-9125-f905150fc1c8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:36.170867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:36.171547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:36.171800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:36.172073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:36.172782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:36.173119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:36.174497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:36.179724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 524/998] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:36.185279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:36.292470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134dfc8-a441-4586-bccf-9c4d89914c4d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:36.328070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:36.328734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:36.329046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:36.329317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:36.330035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:36.330335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:36.331721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:36.341500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 475/999] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:36.342722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:36.406506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:36.445647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9dd5a-dc7b-43e7-818d-962593ad00d7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:36.479147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:36.480174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:36.480454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:36.480726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:36.481552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] RBAC: Adding query filter params (interface=public, service_id=a633525a40414f02b31585d54ff557e0, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:36.481905 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:36.483287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bb208-234d-420e-bbd4-8a28a2445d5e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:36.488658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 525/1000] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:36 2022] GET /identity/v3/endpoints?service_id=a633525a40414f02b31585d54ff557e0&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:36.494210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:36.528614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:36.603971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d172c99-77e9-4ba1-ba66-62ca7fd4afe9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:36.638703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8933-033b-4935-92df-cc04a661672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:36.639216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8933-033b-4935-92df-cc04a661672b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:36.639495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8933-033b-4935-92df-cc04a661672b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:36.639767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8933-033b-4935-92df-cc04a661672b None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:36.640447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8933-033b-4935-92df-cc04a661672b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:36.641856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8933-033b-4935-92df-cc04a661672b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:36.648277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 476/1001] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:36 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:52:37.196187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40563d-4cf0-42f9-9f9a-1b2ba289ae28 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.196737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40563d-4cf0-42f9-9f9a-1b2ba289ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.196737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40563d-4cf0-42f9-9f9a-1b2ba289ae28 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.198311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 526/1002] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:37.203728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.203999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.204298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.221610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:37.222795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:37.293733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 477/1003] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:37.316590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40563d-4cf0-42f9-9f9a-1b2ba289ae28 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:37.425359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40563d-4cf0-42f9-9f9a-1b2ba289ae28 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:37.452436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:37.461515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:37.462600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.463129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.463628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.464479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:37.466107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:37.478624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:37.480001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 28 11:52:37.480854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:37.482553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 527/1004] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Jun 28 11:52:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:37.488473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:37.518903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:37.594677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe526c-6870-4b1e-bfb1-3c3390c0094e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:37.628687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:37.629385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.629657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.629940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.630655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:37.630950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:37.632338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c41be8-ef17-438c-9a99-c85c1e4f241d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:37.637440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 478/1005] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:37.643205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:37.745908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bd72c-9df7-415e-bbd0-f62d1ff0a556 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:37.771688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a4a6-98c5-4ae6-8881-2d50f0c0b92b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.771976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a4a6-98c5-4ae6-8881-2d50f0c0b92b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.772248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a4a6-98c5-4ae6-8881-2d50f0c0b92b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.773657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 479/1006] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:37.778572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.778898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.779290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.789182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:37.789861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.790355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.790786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.791539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:37.791853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:37.793318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:37.798739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 528/1007] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Jun 28 11:52:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:37.803344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:37.804502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:37.804612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:37.871664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:37.873777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 480/1008] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:37.896508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:37.919458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3c3f1-ae51-4381-86c3-46fbeb30961d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:37.920162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:37.957077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:37.957745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:37.958030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:37.958302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:37.959043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:37.960424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61e6d8-d419-461e-be92-8033d53f4ac9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:37.976651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 529/1009] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:38.014896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f612a3b-dae9-4320-a3e7-ae82d1a60add None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:38.061349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:38.062043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:38.062323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:38.062595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:38.063361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:38.064791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba7e7e-ce7a-4f0e-b1cb-c76a5a392abf None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:38.082032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 481/1010] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:37 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:38.695620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3f64f-06d4-4888-b553-1347187af446 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:38.696149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3f64f-06d4-4888-b553-1347187af446 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:38.696149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3f64f-06d4-4888-b553-1347187af446 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:38.697697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 530/1011] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:38.703147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def855de-3662-47a2-a897-016bb5245c8b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:38.703576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def855de-3662-47a2-a897-016bb5245c8b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:38.703746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def855de-3662-47a2-a897-016bb5245c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:38.721030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-def855de-3662-47a2-a897-016bb5245c8b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:38.722239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-def855de-3662-47a2-a897-016bb5245c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:38.805361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 482/1012] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:38.828377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3f64f-06d4-4888-b553-1347187af446 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:38.936176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3f64f-06d4-4888-b553-1347187af446 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:38.953928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:38.976378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:38.977209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:38.977490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:38.977762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:38.978413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:38.980147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:38.992558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:38.993573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:38.994425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 531/1013] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:39.000253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def855de-3662-47a2-a897-016bb5245c8b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:39.003590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:39.103857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def855de-3662-47a2-a897-016bb5245c8b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:39.142860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:39.143840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:39.144310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:39.144748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:39.145642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:39.146117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:39.147661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:39.154013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 483/1014] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:39.159408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:39.269344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37551e2a-66a4-44ed-b055-fbd9bda11480 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:39.304208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:39.304879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:39.305192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:39.305466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:39.306176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:39.306485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:39.307864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789f8ae-cf62-4112-9fd6-2aefb7f2627b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:39.313196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 532/1015] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Jun 28 11:52:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:39.318806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:39.385810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:39.421391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae85477-fe38-4a7e-98fa-afe691df66a4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:39.454763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:39.455439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:39.455718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:39.455991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:39.456744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:39.458192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4ece7-79b6-4af4-9082-7bdcb2da90b2 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:39.476996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 484/1016] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:40.040589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafced69-43aa-40d1-a8d8-66ceb56cce6c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.041128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafced69-43aa-40d1-a8d8-66ceb56cce6c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.041128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafced69-43aa-40d1-a8d8-66ceb56cce6c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.042631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 533/1017] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:40.048026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b957f3-ca46-44d2-811a-1149b7857946 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.048449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b957f3-ca46-44d2-811a-1149b7857946 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.048525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b957f3-ca46-44d2-811a-1149b7857946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.065723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-48b957f3-ca46-44d2-811a-1149b7857946 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:40.066905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-48b957f3-ca46-44d2-811a-1149b7857946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:40.140538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 485/1018] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:40.164042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafced69-43aa-40d1-a8d8-66ceb56cce6c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.164817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d04758-d4be-4704-870f-87542defb84b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.165148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d04758-d4be-4704-870f-87542defb84b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.165423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d04758-d4be-4704-870f-87542defb84b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.166942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 486/1019] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:40.172099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.172382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.172654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.189841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:40.191030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:40.214846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:40.240674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:40.265366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 487/1020] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:40.273133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafced69-43aa-40d1-a8d8-66ceb56cce6c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.288125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.319280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.320003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.320267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.320544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.321767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.323211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.325171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 534/1021] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:40.330822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.401894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4888f-a8db-4512-bd52-02c4c6221c48 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.438540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.438882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc747c0-f658-404a-8e37-603596192e9a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.439349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.439630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.439906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.440596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.442016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.452236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:40.454640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:40.455854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:40.456701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 488/1022] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:40.464386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.480124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.480796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.481128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.481449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.482166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.483615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.512017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:40.512962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 535/1023] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:52:40 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 11:52:40.575828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98697257-dd12-481f-be2c-e4ba2f1516bd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.614903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.615589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.615871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.616143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.616868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] RBAC: Adding query filter params (name=image) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:40.617207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.618616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.629001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 489/1024] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:40.630712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.739843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c48d-db9a-431f-a481-1d52ead50ba2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.776331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.777045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.777323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.777591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.778300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] RBAC: Adding query filter params (type=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:40.778745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.780761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.789402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 536/1025] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:40.795524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.900586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8e0ee-6dbf-44bd-8d3e-a29712712c24 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:40.910581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:40.937742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:40.938407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:40.938686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:40.938956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:40.939737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] RBAC: Adding query filter params (interface=public, service_id=561c272f3d7f4c78adbe74f5b6a536b8, region_id=RegionOne) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:40.940049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:40.941462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b2a94-4163-466b-9315-65c4257234ba None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:40.946857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 490/1026] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/endpoints?service_id=561c272f3d7f4c78adbe74f5b6a536b8&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:40.952567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:40.983559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:41.060484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380a28-bd51-4d3f-abe4-3b800c7c0833 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:41.096588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:41.097293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:41.097574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:41.097847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:41.098529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:41.099908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82880030-b9c5-4d03-ac48-a211d0b07528 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:41.107395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 537/1027] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:52:40 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:52:41.668183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3389-4b87-486c-beb7-b08676da7688 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:41.668810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3389-4b87-486c-beb7-b08676da7688 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:41.668810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3389-4b87-486c-beb7-b08676da7688 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:41.670495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 491/1028] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:41.675949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:41.676246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:41.676523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:41.695301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:41.696522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:41.784967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 538/1029] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:41.804639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce3389-4b87-486c-beb7-b08676da7688 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:41.910879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce3389-4b87-486c-beb7-b08676da7688 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:41.944384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:41.945165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:41.945433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:41.945715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:41.956405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:41.964432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:41.965402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:41.966289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:41.969261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 492/1030] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:41.971666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:41.996585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.083496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1df8e-a10c-464c-9ee3-04d2efb24463 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.119259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.119937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.120220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.120493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.121270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:42.121579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.123005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.131552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 539/1031] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:42.137113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.169428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.242654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d10e2-2146-4235-8831-f4252596b1ff None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.276959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.277778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.278065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.278326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.289121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.290572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:42.307894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:42.308791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:42.310087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:42.310087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:42.310087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:42.310087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:42.310906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 493/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:42.315129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.400126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.418420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4b96-6ae2-4745-8ad2-bbb5ce09006d None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.452077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.452739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.453047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.453318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.454039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:42.454345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.455722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c8db8-0f85-46df-ba46-02790740f164 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.468202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 540/1033] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:42.473688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.563209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.581241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35b0f-6d12-43ab-beac-59a001f404e6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.615445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.616245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.616512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.616799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.624859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.626338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:42.640728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:42.641887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:42.642872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 494/1034] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:42.649071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.649691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3461a-0e48-4ddd-9790-923a0e778f83 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.649995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3461a-0e48-4ddd-9790-923a0e778f83 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.650231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3461a-0e48-4ddd-9790-923a0e778f83 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.651768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 541/1035] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:42.657377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.657629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.657924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.675822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:42.677099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:42.749030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 542/1036] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:42.760165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbe896-c7f4-4153-b29a-b3c8928dfbec None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.774299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.804621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.805439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.805713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.805994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.806816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:42.807117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.808763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.833752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 495/1037] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:42.835715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.838768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.858121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:42.885657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafc555-1b9b-4a01-b02a-9392df9347c6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.921718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.922640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.922924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.923255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:42.924563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:42.926072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:42.932571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 543/1038] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:42.933542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:42.946511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00076a81-4574-4082-a846-bc67ce3ac53f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:42.989806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:42.990488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:42.990763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:42.991030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.049891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc958266-7f44-41b2-9ebc-e7cccc23a5d0 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.063287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.064746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.066857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.092256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.093014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.093258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.093452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.094147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.095590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd16b-3b70-497f-823b-d25420638019 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.114199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.122916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 544/1039] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:52:42 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:43.124408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c8d2-d654-491c-aba5-e23d69a6da36 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.124697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c8d2-d654-491c-aba5-e23d69a6da36 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.124999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c8d2-d654-491c-aba5-e23d69a6da36 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.126836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 545/1040] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:43.132955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.133239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.133515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.151679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:43.152931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:43.185319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 496/1041] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:42 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:43.187699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.241119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 546/1042] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:43.260451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.300259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb202-7704-4652-9183-1acccf00edcb None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.302017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.341090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.341908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.342189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.342521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.354968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.356440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.372616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef9b0d-5cf9-4289-8e92-d4596585cfc4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:43.375062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:43.375967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:43.376865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:43.376865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:43.376865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:43.376865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:43.376865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 497/1043] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:43.382006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.403323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.412725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.413560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.413862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.414133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] PATH_INFO: `/v3/services/image` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.414783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.416169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:43.428726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:43.429806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:43.434458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 547/1044] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:43.436366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.496088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9870e5-0bb6-4fe2-bab4-5063f893d6b4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.532996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.533548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.533833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.534108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.534837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:43.535144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.536548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.547064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed917f1-d787-4b93-8b02-a031708d746e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.552757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 498/1045] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:43.553661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.570870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.587090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.587790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.588053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.588326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.589068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:43.589376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.590758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.596008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 548/1046] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:43.601519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.662305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.670089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdb892-77eb-457f-8ee7-a79024054b32 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.704402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.705240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.705517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.705786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.710908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8fff-1b28-4064-850a-b6b20bb3d0b9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.712648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.714142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:43.728530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:43.729939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:43.730838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:43.733254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 499/1047] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:43.735915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.750960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.751607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.751886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.752158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.752889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] RBAC: Adding query filter params (type=image) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:43.753212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.754577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.759742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 549/1048] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:43.766160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.823558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.856059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ef308-d88c-49c1-9e0d-333061965e4c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.881076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8ad04-3200-4db2-9cca-327e7e150c43 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:43.893319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.894103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.894369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.894647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.895557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:43.895910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.897468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.901038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:43.911672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 500/1049] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:43.917620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:43.924536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:43.925323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:43.925610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:43.925888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:43.926634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:43.928052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bbc33-8951-4dc5-9634-050833ec43e5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:43.956940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 550/1050] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:52:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:44.039892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563479f5-2f0b-4463-9be9-83002ba00141 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:44.077491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:44.086259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:44.086935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:44.087202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:44.087478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:44.088313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] RBAC: Adding query filter params (role.id=ceee4461d8df498da7cdc5d10c0c9619, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:44.088625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:44.090136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daeefa-ec57-4f65-94d4-dfdd116eba00 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:44.097203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 501/1051] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:43 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=ceee4461d8df498da7cdc5d10c0c9619 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:45.289369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e0f4-d6f1-40b8-a0f5-8b1dc7b832d3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:45.289909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e0f4-d6f1-40b8-a0f5-8b1dc7b832d3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:45.289909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0e0f4-d6f1-40b8-a0f5-8b1dc7b832d3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:45.291509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 551/1052] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:45.296726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:45.297178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:45.297336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:45.314448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:45.315706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:45.388962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 502/1053] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:45.411511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e0f4-d6f1-40b8-a0f5-8b1dc7b832d3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:45.516834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0e0f4-d6f1-40b8-a0f5-8b1dc7b832d3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:45.552176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:45.553252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:45.553683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:45.554122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:45.565659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:45.573686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:45.574689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:45.576687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:45.580918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 552/1054] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:45.582205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:45.685432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ab86-ae7b-4210-b9b7-bdacc9fcf798 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:45.709084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:45.721827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:45.722514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:45.722799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:45.723074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:45.723830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:45.724141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:45.725565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:45.740220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 503/1055] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:45.741652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:45.791227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:45.856065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03e29e-db62-4ffc-83d3-0711a8429cab None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:45.893298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:45.894183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:45.894464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:45.894752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] PATH_INFO: `/v3/users/c2e579d4d3eb4ba29e291d5770f2628a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:45.909114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] RBAC: Authorizing `identity:get_user(user_id=c2e579d4d3eb4ba29e291d5770f2628a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:45.910578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc6e14-8136-4c87-865d-80e26c6cf105 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:45.912502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 553/1056] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:52:45 2022] GET /identity/v3/users/c2e579d4d3eb4ba29e291d5770f2628a => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:45.918188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.021746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0849-0532-454a-b67e-9a0fa8f0b889 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.050597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579db3-43f6-4add-8dc9-043555ef4267 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.050902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579db3-43f6-4add-8dc9-043555ef4267 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.051130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579db3-43f6-4add-8dc9-043555ef4267 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.052677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 554/1057] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:46.056943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.057417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245342-3e34-4eda-87c4-b5897aeeb8c9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.057632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.057740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245342-3e34-4eda-87c4-b5897aeeb8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.057893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.057999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245342-3e34-4eda-87c4-b5897aeeb8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.058148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.074943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-eb245342-3e34-4eda-87c4-b5897aeeb8c9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:46.076757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-eb245342-3e34-4eda-87c4-b5897aeeb8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:46.115679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.129610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.131089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.144653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 555/1058] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:46.146257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e15834-1533-4345-bd6f-158047f3df99 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.146552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e15834-1533-4345-bd6f-158047f3df99 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.147075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e15834-1533-4345-bd6f-158047f3df99 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.148227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 556/1059] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:46.153482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.153797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.154104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.173718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:46.174924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:46.192083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.235024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 504/1060] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:46.238086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.241963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 557/1061] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:46.243048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.320149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.337677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53a1d1-3b08-4c1b-9f7a-53baee75c837 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.343604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f317870-e8c7-47f3-b48a-a3b4c0aae03e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.373011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.373620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.373914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.374177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.379260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.380132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.380419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.380689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.381897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.383533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.385299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 558/1062] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:46.386052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.410824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.445051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.446529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.486421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ba5f-bfb5-4f01-b6e7-554e7c9dca5c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.520315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.521168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.521452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.521728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.529689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.532171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:46.540323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:46.541307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:46.542317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 559/1063] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:46.543512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.548836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 505/1064] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:46.549690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.628994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.649898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bcab4-f60e-49d6-89ca-caa17ebe7e35 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.650521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcdef9-3635-41dc-97ca-bb4f4af2bda8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.683413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.684069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.684348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.684622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.685543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=fdec5033904f438888fea242a78f59e6, user.id=c2e579d4d3eb4ba29e291d5770f2628a) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:46.685855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.686415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.687082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.687285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.687376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.687636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.688369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:46.688668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.690059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.692673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 560/1065] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/role_assignments?user.id=c2e579d4d3eb4ba29e291d5770f2628a&scope.project.id=fdec5033904f438888fea242a78f59e6&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:46.693609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.699169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 506/1066] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:46.705432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.763456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.814287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ecb1b-3d26-4b63-8f08-28b9f71375e5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.826848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75643b47-e9df-45a1-b078-6c98c6e182b7 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:46.850951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.851764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.852046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.852316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.864303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.865790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.867510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:46.868332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:46.868608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:46.868878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:46.873325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:46.880271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:46.881699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:46.886293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:46.887176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:46.888193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:46.888193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:46.888193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:46.888193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:46.895525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 561/1067] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:46.896349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:46.898903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:46.899783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:46.905662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:46.905662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:46.905662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:46.905662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:46.905662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 507/1068] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:46.907522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.015731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:47.018064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f99db-f773-4243-98b5-3f59c6aead33 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.019028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04985db-ea34-4903-b774-4c266ed2fe6e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.056520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.057217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.057466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.057754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.058478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:47.058779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.060112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.060692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.060821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.061078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.061349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.062357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:47.062666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.064570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.073163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 508/1069] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:47.079040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.079619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 562/1070] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:47.190680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aefa6a-2e31-4564-88d9-f0a78db9d5a4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.231123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.232332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.232620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.232890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.240953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.243235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:47.259250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:47.260217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:47.261964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 509/1071] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:47.267505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.329084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:47.371308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963210b-96bd-4eb1-9978-5ce20e07eaf8 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.405966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.406618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.406896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.407216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.408037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:47.408341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.409821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.426954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 563/1072] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:47.432706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.456262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:47.536529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a125e9d-6db5-4f59-86ab-85c0df0b45b0 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.571196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.571977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.572252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.572518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.582899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.584339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:47.602966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:47.603858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:47.605175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:47.605175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:47.605175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:47.605175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:47.605175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 510/1073] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:47.611009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.714428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9a474-4a35-438f-a75c-149215b7f150 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.731112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:47.751910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.752581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.752863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.753166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.753874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:47.754169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.755557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.767308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 564/1074] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:47.772819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:47.846378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:47.878083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0255672-1313-4e6a-a94f-9174ee2429a3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:47.913127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:47.913921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:47.914200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:47.914468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:47.921511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:47.922965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:47.936683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:47.937621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:47.938524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 511/1075] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:47.944063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:48.050380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c0ed-522f-4355-900b-fae9fd6836bb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:48.084822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:48.085503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:48.085776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:48.086052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:48.086833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:48.087133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:48.088573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd3df-7e6a-4ebe-8d88-f6fb57950266 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:48.100930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 565/1076] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:48.107917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:48.210569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea338-afc1-483e-9c1e-f3bed3c53bbe None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:48.241253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:48.248025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:48.248945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:48.249240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:48.249512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/ceee4461d8df498da7cdc5d10c0c9619` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:48.251858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=77cc74fa2a6c48d0b85e3b2e70f451f5, role_id=ceee4461d8df498da7cdc5d10c0c9619)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:48.253362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b483a5a-05d3-4b62-abe1-7db94acd8e3b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:48.267397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 512/1077] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:48 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/77cc74fa2a6c48d0b85e3b2e70f451f5/roles/ceee4461d8df498da7cdc5d10c0c9619 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:52:48.724136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c51-7b1e-4897-9873-aae42b342f85 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:48.724697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c51-7b1e-4897-9873-aae42b342f85 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:48.724697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c51-7b1e-4897-9873-aae42b342f85 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:48.726154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 566/1078] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:48.732212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:48.732647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:48.732732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:48.750315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:48.751496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:48.821943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 513/1079] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:48.844829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b2c51-7b1e-4897-9873-aae42b342f85 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:48.885690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:48.953669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b2c51-7b1e-4897-9873-aae42b342f85 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:48.994838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:48.995556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:48.995837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:48.996112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:49.007022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:49.014750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:49.015677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:49.016529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 567/1080] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:49.022470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:49.125828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e6197-b091-4272-92a4-8e8b590cbdb6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:49.148522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:49.162042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:49.162708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:49.162989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:49.163265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:49.164017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:49.164323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:49.165739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:49.174070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 514/1081] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:49.179967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:49.294346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:49.301650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5981e-b292-4951-9802-c7a2f61df3b1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:49.340346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:49.341754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:49.342039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:49.342310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] PATH_INFO: `/v3/users/c2e579d4d3eb4ba29e291d5770f2628a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:49.343510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=c2e579d4d3eb4ba29e291d5770f2628a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:49.344972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:49.346613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 568/1082] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:52:49 2022] GET /identity/v3/users/c2e579d4d3eb4ba29e291d5770f2628a => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:49.352585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:49.460467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e385f-6d87-47ed-8fd8-23131ca1791b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:49.495583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:49.496449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:49.496721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:49.497010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:49.499105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:49.500550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a514f8-d801-4695-aa07-638dbb764c04 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:49.502284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 515/1083] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:52:49 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:49.508364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:49.612334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d2240-3f78-4c16-abe5-943a791f8eb9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:49.646875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:49.647799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:49.648052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:49.648354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6/users/c2e579d4d3eb4ba29e291d5770f2628a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:49.650516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdec5033904f438888fea242a78f59e6, user_id=c2e579d4d3eb4ba29e291d5770f2628a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:49.651975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e945b-5b8d-48c5-8cdb-ea562bbbdd23 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:49.667012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 569/1084] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:52:49 2022] PUT /identity/v3/projects/fdec5033904f438888fea242a78f59e6/users/c2e579d4d3eb4ba29e291d5770f2628a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 11:52:50.310890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cbece-54b5-4557-a4ae-b15739feb0b4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:50.311451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cbece-54b5-4557-a4ae-b15739feb0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:50.311451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cbece-54b5-4557-a4ae-b15739feb0b4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:50.312824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 516/1085] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:50.318082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:50.318504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:50.318589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:50.335980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:50.337213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:50.355829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:50.406704 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 570/1086] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:50.430534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cbece-54b5-4557-a4ae-b15739feb0b4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:50.486654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:50.535739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cbece-54b5-4557-a4ae-b15739feb0b4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:50.570245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:50.570997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:50.571274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:50.571548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:50.582488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:50.590515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:50.591587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 28 11:52:50.592512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:50.593185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 517/1087] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Jun 28 11:52:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:50.598771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:50.706712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328754d-d05f-440b-9140-8e5981906d11 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:50.742082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:50.742764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:50.743049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:50.743307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:50.744066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:50.744371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:50.745801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:50.755376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 571/1088] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:52:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:50.761603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:50.868626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e503c-6685-4275-977f-0905dbd9fdb6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:50.881419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:50.902503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:50.903296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:50.903584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:50.903849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:50.914129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:50.915580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:50.932642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:50.933619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:50.934691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:50.934691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:50.934691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:50.934691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:50.934691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 518/1089] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:50.939972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:50.960306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.054080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56dce3-0577-48b1-9b5e-cb9921a4337b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.088893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.089591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.089871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.090143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.090851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:51.091178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:51.092560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:51.104732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 572/1090] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:51.110757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.213940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e927967-f551-40e8-9ea0-2626967e824b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.249928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.250726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.251005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.251280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.258639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:51.260083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:52:51.275076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:51.276258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:51.277840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 519/1091] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:52:51.284108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.369077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.394430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b6a3-7575-4f33-8486-c7a15f1f8b91 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.429399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.430065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.430339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.430606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.431430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:51.431741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:51.433198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44614db-b857-4dac-b0ff-d329cf1026e3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:51.451593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 573/1092] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:51.457179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.485076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.563715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6ade3-7407-4a7a-a88d-13d996208da3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.601465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.602124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.602405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.602674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.674043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:51.675487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:51.677398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5b09-9f0f-4453-b3ec-cd6bf86ba0eb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.677684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5b09-9f0f-4453-b3ec-cd6bf86ba0eb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.678000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d5b09-9f0f-4453-b3ec-cd6bf86ba0eb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.679337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 574/1093] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:52:51.684221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.684503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.684808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.701452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.701640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:51.702833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:51.769961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 575/1094] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:52:51.786735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 520/1095] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:51.787809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.793296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.850610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.890563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbff72-af41-47fa-b5ff-0740851b641a None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.891648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a205f5-0b10-4e5b-b43c-5e0b017c2718 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:51.913894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:51.925189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.925998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.926282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.926557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.927441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:51.928241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:51.928521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:51.928797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:51.936572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:52:51.939341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:51.940795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:52:51.944227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:51.945351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:52:51.946371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 521/1096] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:51.951540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:52:51.958807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:51.959743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:51.960781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:52:51.960781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:52:51.960781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:52:51.960781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:51.960781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 576/1097] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:51.965549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.052825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506801f-819b-4830-bf0d-12b1bdd9e8ca None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.063823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d1a44-e40c-4c0d-8b3c-f64911356886 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.081535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.088774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.089483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.089766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.090041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.090800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:52.091163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.092540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.100876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.101441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 522/1098] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:52.101545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.101837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.102106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.102822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:52.103131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.104838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.107642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.121401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 577/1099] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:52.122483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.152774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.205605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62808092-e565-4b2f-8a12-bf1a8466710b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.221622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e64b3-d9a3-49d0-a844-8ced8241b2c3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.239386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.240264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.240546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.240816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] PATH_INFO: `/v3/users/c2e579d4d3eb4ba29e291d5770f2628a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.242134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] RBAC: Authorizing `identity:get_user(user_id=c2e579d4d3eb4ba29e291d5770f2628a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.243594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.245349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 523/1100] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/users/c2e579d4d3eb4ba29e291d5770f2628a => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:52.251164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.256364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.257205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.257487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.257758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.264692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.266192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:52:52.280998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:52:52.281878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:52:52.282784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 578/1101] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:52:52.288316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.323553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.352628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0f3-5939-4179-93ec-79a293b8582b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.367818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.388666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.389369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.389652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.389925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.393586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fe02-4a21-4484-9c4e-82e2e19e0394 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.428887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.429599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.429873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.430140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.430915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:52.431206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.432648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.444482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 579/1102] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:52:52.450727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.459288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.460744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.545406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.551656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b8a15-fe45-495c-a219-512dd653e554 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.564154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 524/1103] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:52.570188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.584434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:52.592511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.593224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.593564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.593894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.594711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] RBAC: Adding query filter params (role.id=ceee4461d8df498da7cdc5d10c0c9619, scope.project.id=5709e926be584e859153090dd261a990, user.id=77cc74fa2a6c48d0b85e3b2e70f451f5) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:52.595039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.596698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.603814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 580/1104] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/role_assignments?user.id=77cc74fa2a6c48d0b85e3b2e70f451f5&scope.project.id=5709e926be584e859153090dd261a990&role.id=ceee4461d8df498da7cdc5d10c0c9619 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:52.678091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ef63-24b1-45cb-ab6e-036cb62c8b33 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.716052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.716944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.717257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.717532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.719062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.720546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f52e0a-1d4d-450b-9377-af40fc20a6a0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.722368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 525/1105] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:52.728801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:52.839282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5a8b4-dde8-4707-b373-a887d5e96843 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:52.873547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:52.874215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:52.874495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:52.874764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:52.875607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=fdec5033904f438888fea242a78f59e6, user.id=c2e579d4d3eb4ba29e291d5770f2628a) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:52.875922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:52.877402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049778d-165c-4143-882a-9866d48fd280 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:52.883488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 581/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:52 2022] GET /identity/v3/role_assignments?user.id=c2e579d4d3eb4ba29e291d5770f2628a&scope.project.id=fdec5033904f438888fea242a78f59e6&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:54.923187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77e23-8c2f-420e-a272-253456126754 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:54.924128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77e23-8c2f-420e-a272-253456126754 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:54.924128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77e23-8c2f-420e-a272-253456126754 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:54.926386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 526/1107] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:54.932661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:54.933466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:54.933605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:54.951774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:54.953012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:54.971003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:55.027078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 582/1108] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:55.052786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff77e23-8c2f-420e-a272-253456126754 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:55.144730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:55.165746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff77e23-8c2f-420e-a272-253456126754 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:55.206807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:55.207598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:55.207875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:55.208149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:55.209486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:55.211202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb9828-bc66-4b0f-9646-897936ba9d46 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:55.212824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 527/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:52:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:55.219809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:55.330142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129354c-ecf4-44a0-8f35-05ad9ffc1960 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:55.375969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:55.376936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:55.377228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:55.377522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:55.378576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:55.380783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1595dd7-263a-465c-8f81-e46703d7b7ec None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:55.405185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 583/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:52:55 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:52:57.553534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f4de-fb48-43a2-9f9a-556017a30fa9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:57.554131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f4de-fb48-43a2-9f9a-556017a30fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:57.554131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8f4de-fb48-43a2-9f9a-556017a30fa9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:57.555477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 528/1111] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:52:57.561051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:57.561503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:57.561599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:57.579903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:52:57.581115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:52:57.583892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:57.656989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 584/1112] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:52:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:52:57.681123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8f4de-fb48-43a2-9f9a-556017a30fa9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:57.796288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8f4de-fb48-43a2-9f9a-556017a30fa9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:57.849786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:57.850574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:57.850848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:57.851115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] PATH_INFO: `/v3/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:57.852779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] RBAC: Authorizing `identity:get_role(role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:57.854864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:57.856264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 529/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:52:57 2022] GET /identity/v3/roles/a0087dfad30d4fd095fbec15fc10286d => generated 223 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:57.862722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:57.974010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e007a3-71e8-429e-9cf6-7068fbc2531d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:58.009226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:58.010097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:58.010377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:58.010649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] PATH_INFO: `/v3/users/52d30ee79e6f4167a85b585b2196a109` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:58.021213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:58.024799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] RBAC: Authorizing `identity:get_user(user_id=52d30ee79e6f4167a85b585b2196a109)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:58.026335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:58.028028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 585/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:52:57 2022] GET /identity/v3/users/52d30ee79e6f4167a85b585b2196a109 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:58.034659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:58.064714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:58.153962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723b75-0886-4a1a-9348-821729f9556f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:58.190469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:58.191117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:58.191396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:58.191669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:58.287738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:58.289228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:58.313520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:58.406552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 530/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:52:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:52:58.415022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:58.518131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485a28-11cb-4070-8d82-862f45f4c553 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:58.562899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:58.563814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:58.564091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:58.564393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:58.565860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:58.567321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c11c67-13a5-4da3-88d4-0ade2d180be5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:58.569094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 586/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:52:58 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:52:58.576616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:52:58.680458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073287d2-fc4d-477b-baa0-af463a07eb46 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:52:58.698624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:52:58.717745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:52:58.718395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:52:58.718677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:52:58.718948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:52:58.719777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=fdec5033904f438888fea242a78f59e6, user.id=52d30ee79e6f4167a85b585b2196a109) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:52:58.720088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:52:58.721558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b26fa-8e95-4061-9cba-6c44190b83d5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:52:58.727084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 531/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:52:58 2022] GET /identity/v3/role_assignments?user.id=52d30ee79e6f4167a85b585b2196a109&scope.project.id=fdec5033904f438888fea242a78f59e6&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:00.778532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa1c0d-c809-4274-a15a-c27c1810bdcd None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:00.779328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa1c0d-c809-4274-a15a-c27c1810bdcd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:00.779328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa1c0d-c809-4274-a15a-c27c1810bdcd None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:00.781392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 587/1118] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:00.787630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:00.788094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:00.788193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:00.808405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:00.809779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:00.888251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 532/1119] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:00.911630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa1c0d-c809-4274-a15a-c27c1810bdcd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:00.984011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:01.022166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa1c0d-c809-4274-a15a-c27c1810bdcd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:01.059045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:01.059968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:01.060245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:01.060516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] PATH_INFO: `/v3/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:01.062038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] RBAC: Authorizing `identity:get_role(role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:01.063416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:01.064790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 588/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:00 2022] GET /identity/v3/roles/a0087dfad30d4fd095fbec15fc10286d => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:01.070904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:01.178256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffb12f-1298-4420-a02f-d87852ab18b7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:01.192118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:01.215643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:01.216579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:01.216860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:01.217163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] PATH_INFO: `/v3/users/52d30ee79e6f4167a85b585b2196a109` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:01.218405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] RBAC: Authorizing `identity:get_user(user_id=52d30ee79e6f4167a85b585b2196a109)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:01.219859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:01.221518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 533/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:01 2022] GET /identity/v3/users/52d30ee79e6f4167a85b585b2196a109 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:01.231538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:01.339743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c82-4cf0-473f-a8bb-a80c7c237859 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:01.375460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:01.376306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:01.376586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:01.376858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:01.378192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:01.379628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf51cc8-21d8-4a88-b09a-59dd356f3893 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:01.381373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 589/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:01 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:01.387834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:01.495437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33454-0c32-4fbf-8cbb-8f7ff6ae9444 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:01.529883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:01.530768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:01.531053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:01.531316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6/users/52d30ee79e6f4167a85b585b2196a109/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:01.533688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] RBAC: Authorizing `identity:create_grant(project_id=fdec5033904f438888fea242a78f59e6, user_id=52d30ee79e6f4167a85b585b2196a109, role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:01.535135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c254e-2368-46a5-a97f-d22fe4db10df None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:01.550495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 534/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:53:01 2022] PUT /identity/v3/projects/fdec5033904f438888fea242a78f59e6/users/52d30ee79e6f4167a85b585b2196a109/roles/a0087dfad30d4fd095fbec15fc10286d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:53:03.554745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ac5-2307-4d25-9353-f5c5991e10cb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:03.555313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ac5-2307-4d25-9353-f5c5991e10cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:03.555313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a2ac5-2307-4d25-9353-f5c5991e10cb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:03.556685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 590/1124] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:53:03.563305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:03.564060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:03.564213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:03.592200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:03.593950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:03.636130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:03.670996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 535/1125] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:03.694963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ac5-2307-4d25-9353-f5c5991e10cb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:03.699516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:03.806356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a2ac5-2307-4d25-9353-f5c5991e10cb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:03.849909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:03.850769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:03.851051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:03.851327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] PATH_INFO: `/v3/roles/a0087dfad30d4fd095fbec15fc10286d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:03.852949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] RBAC: Authorizing `identity:get_role(role_id=a0087dfad30d4fd095fbec15fc10286d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:03.854706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:03.856104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 591/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:03 2022] GET /identity/v3/roles/a0087dfad30d4fd095fbec15fc10286d => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:03.864939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:03.976108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6e263-8e38-421f-92d0-c7c248e4aca2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:04.026146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:04.027403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:04.027790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:04.028176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] PATH_INFO: `/v3/users/52d30ee79e6f4167a85b585b2196a109` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:04.030070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] RBAC: Authorizing `identity:get_user(user_id=52d30ee79e6f4167a85b585b2196a109)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:04.031540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:04.033229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 536/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:03 2022] GET /identity/v3/users/52d30ee79e6f4167a85b585b2196a109 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:04.039403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:04.111684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:04.147219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7df4c-9584-4c07-85fe-065bd7b2483a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:04.183325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:04.183960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:04.184243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:04.184567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:04.253103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:04.254555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:04.327484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:04.363654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 592/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 28 11:53:04.369689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:04.412917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:04.474349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09940e60-4c03-41f3-b902-4761e4b93c6e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:04.510442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:04.511295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:04.511566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:04.511833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] PATH_INFO: `/v3/projects/fdec5033904f438888fea242a78f59e6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:04.513623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] RBAC: Authorizing `identity:get_project(project_id=fdec5033904f438888fea242a78f59e6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:04.515071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4b4f7-67f4-480e-9d43-c5d48837c3ae None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:04.516758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 537/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:04 2022] GET /identity/v3/projects/fdec5033904f438888fea242a78f59e6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:04.523108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:04.643670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acd20-fd1d-4127-a8f4-9a10f4111b02 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:04.678887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:04.679546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:04.679830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:04.680101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:04.680952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] RBAC: Adding query filter params (role.id=a0087dfad30d4fd095fbec15fc10286d, scope.project.id=fdec5033904f438888fea242a78f59e6, user.id=52d30ee79e6f4167a85b585b2196a109) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:04.681311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:04.682752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25df609-37bc-4bb4-95aa-1efc48239ae7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:04.688831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 593/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:53:04 2022] GET /identity/v3/role_assignments?user.id=52d30ee79e6f4167a85b585b2196a109&scope.project.id=fdec5033904f438888fea242a78f59e6&role.id=a0087dfad30d4fd095fbec15fc10286d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:06.769782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14cc70-24ef-4d32-9bc9-8a76079b9c5a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:06.770336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14cc70-24ef-4d32-9bc9-8a76079b9c5a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:06.770336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14cc70-24ef-4d32-9bc9-8a76079b9c5a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:06.772425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 538/1131] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:06.781028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:06.781502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:06.781607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:06.804511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:06.805752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:06.836063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:06.887218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 594/1132] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:53:06.911738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14cc70-24ef-4d32-9bc9-8a76079b9c5a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:07.035834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14cc70-24ef-4d32-9bc9-8a76079b9c5a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:07.042925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:07.070569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:07.071359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:07.071632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:07.071908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:07.073127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:07.074579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab2e5b-5535-4b83-a1bc-1a2b79aabad6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:07.076226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 539/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:53:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:07.085092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:07.192341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d932bc2-8c0f-4be8-9326-080ac8f2d50a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:07.233355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:07.234357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:07.234778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:07.235185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:07.236288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:07.238644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49b1e7-1ce4-4c30-913b-1e428c8efabe None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:07.271176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 595/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:53:07 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:53:09.355919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7dc10-345f-4ba5-af38-59c18688501f None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:09.356522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7dc10-345f-4ba5-af38-59c18688501f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:09.356522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7dc10-345f-4ba5-af38-59c18688501f None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:09.358341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 540/1135] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:09.364704 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:09.365433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:09.365650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:09.386633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:09.387836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:09.398930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:09.465674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 596/1136] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:09.504804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7dc10-345f-4ba5-af38-59c18688501f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:09.625472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7dc10-345f-4ba5-af38-59c18688501f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:09.661122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:09.661911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:09.662195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:09.662471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:09.663783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:09.665284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724ac8c-d99d-478f-bbc4-7574035d71e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:09.666907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 541/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:53:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:09.673758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:09.801218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d627768-f510-4c84-8788-03dfa5642aaa None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:09.838175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:09.838838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:09.839110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:09.839377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:09.840072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:09.842066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd506592-814f-45ac-9d67-7ac0e60cfa7e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:09.870875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 597/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:53:09 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:53:11.947904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c954-0397-4c95-868f-d95b4eeb3960 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:11.948465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c954-0397-4c95-868f-d95b4eeb3960 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:11.948465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c954-0397-4c95-868f-d95b4eeb3960 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:11.949841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 542/1139] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:11.955850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:11.956296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:11.956395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:11.962588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:11.978036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:11.979571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:12.050872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 598/1140] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:12.074884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0c954-0397-4c95-868f-d95b4eeb3960 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:12.122965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:12.194759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0c954-0397-4c95-868f-d95b4eeb3960 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:12.238898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:12.239705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:12.239990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:12.240262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:12.251933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:12.259585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:12.260825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:12.261792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 543/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:12.267367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:12.384300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c542b6-0495-4982-81b8-3e29791ce51d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:12.415474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:12.423534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:12.424540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:12.424963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:12.425335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:12.426587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:12.427057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:12.429268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:12.442445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 599/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:12.448384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:12.561336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761b04b-5ecf-4698-b786-91382f33efbd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:12.562986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:12.596925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:12.597826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:12.598109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:12.598382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] PATH_INFO: `/v3/users/2ad900f5d797453e94107cd46bf61c8a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:12.613782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] RBAC: Authorizing `identity:get_user(user_id=2ad900f5d797453e94107cd46bf61c8a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:12.615245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:12.616936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 544/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:12 2022] GET /identity/v3/users/2ad900f5d797453e94107cd46bf61c8a => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:12.623444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:12.741197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400291f3-3797-4a58-90bf-9b45ae895be4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:12.780003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:12.780654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:12.780950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:12.781228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:12.842679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:12.852775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:12.854676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:12.972054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 600/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:53:12.978172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:13.083995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f653c0-cca0-472c-bc8a-5c1ec3b4faf5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:13.119634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:13.120481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:13.120758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:13.121055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] PATH_INFO: `/v3/projects/ffcc7c72e12440e69aa054c52ea50968` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:13.122344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] RBAC: Authorizing `identity:get_project(project_id=ffcc7c72e12440e69aa054c52ea50968)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:13.124007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339c77-e13a-419d-ae56-5f18a87a9c3d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:13.125746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 545/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:12 2022] GET /identity/v3/projects/ffcc7c72e12440e69aa054c52ea50968 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:13.132431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:13.235534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:13.245347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24a87-b97d-4980-9566-3b958c041788 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:13.283230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:13.283887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:13.284165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:13.284436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:13.285296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=ffcc7c72e12440e69aa054c52ea50968, user.id=2ad900f5d797453e94107cd46bf61c8a) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:13.285604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:13.287042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f3f9-2b04-48b7-97e7-91469a88cd13 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:13.292677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 601/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:53:13 2022] GET /identity/v3/role_assignments?user.id=2ad900f5d797453e94107cd46bf61c8a&scope.project.id=ffcc7c72e12440e69aa054c52ea50968&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:15.306243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657fd5f-15b7-46ff-a13a-b423d073ee04 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:15.306810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657fd5f-15b7-46ff-a13a-b423d073ee04 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:15.306810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657fd5f-15b7-46ff-a13a-b423d073ee04 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:15.308949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 546/1147] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:15.317970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:15.318386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:15.318556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:15.339100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:15.340301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:15.413521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 602/1148] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:15.437561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657fd5f-15b7-46ff-a13a-b423d073ee04 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:15.470042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:15.548059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657fd5f-15b7-46ff-a13a-b423d073ee04 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:15.583071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:15.583786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:15.584056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:15.584319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:15.595449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:15.602922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:15.604342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:15.605385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 547/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:15.613498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:15.712399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:15.724876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d26ae0-3efe-4e54-be8b-2ac2c4a3ca23 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:15.759848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:15.760517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:15.760799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:15.761102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:15.761851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:15.762157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:15.763565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:15.774830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 603/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:15.781186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:15.870683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:15.890885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591449e9-fe9a-4478-aaf3-5b8ee03495fa None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:15.932393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:15.933318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:15.933599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:15.933875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] PATH_INFO: `/v3/users/2ad900f5d797453e94107cd46bf61c8a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:15.935152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] RBAC: Authorizing `identity:get_user(user_id=2ad900f5d797453e94107cd46bf61c8a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:15.936779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:15.938419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 548/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:15 2022] GET /identity/v3/users/2ad900f5d797453e94107cd46bf61c8a => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:15.947219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:16.072527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80394bfa-92ab-4443-83e0-602917a5c20e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:16.115172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:16.116040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:16.116321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:16.116592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] PATH_INFO: `/v3/projects/ffcc7c72e12440e69aa054c52ea50968` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:16.117865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ffcc7c72e12440e69aa054c52ea50968)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:16.119320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882cf2-c6a6-44f3-af9a-2ddcd45ff4a7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:16.121120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 604/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:15 2022] GET /identity/v3/projects/ffcc7c72e12440e69aa054c52ea50968 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:16.127410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:16.232375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efff59-1ff9-407e-b7dd-b2249fe2a4cd None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:16.267009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:16.267898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:16.268180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:16.268453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] PATH_INFO: `/v3/projects/ffcc7c72e12440e69aa054c52ea50968/users/2ad900f5d797453e94107cd46bf61c8a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:16.270575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffcc7c72e12440e69aa054c52ea50968, user_id=2ad900f5d797453e94107cd46bf61c8a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:16.272026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00436a-72a8-4b3e-9481-4eaefbf9d992 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:16.288294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 549/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:53:16 2022] PUT /identity/v3/projects/ffcc7c72e12440e69aa054c52ea50968/users/2ad900f5d797453e94107cd46bf61c8a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:53:18.306655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1786682-fdbc-4d91-a0c7-a19a3c940030 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:18.307506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1786682-fdbc-4d91-a0c7-a19a3c940030 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:18.307506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1786682-fdbc-4d91-a0c7-a19a3c940030 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:18.309825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 605/1154] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:18.315418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:18.315836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:18.315954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:18.334550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:18.335712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:18.346759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:18.406079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 550/1155] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:18.430717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1786682-fdbc-4d91-a0c7-a19a3c940030 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:18.458096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:18.547990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1786682-fdbc-4d91-a0c7-a19a3c940030 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:18.583338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:18.584052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:18.584326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:18.584593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:18.596194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:18.603621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:18.605025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:53:18.605902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 606/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:18.611530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:18.714502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702abef8-2183-4cd0-8eee-a3655e246a89 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:18.751413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:18.752068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:18.752343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:18.752607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:18.753382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:18.753692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:18.755065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:18.763766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 551/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:18.769775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:18.856043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:18.881139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627837-38f2-4506-9a81-38759b3fe3a1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:18.916391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:18.917301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:18.917578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:18.917844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] PATH_INFO: `/v3/users/2ad900f5d797453e94107cd46bf61c8a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:18.919611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] RBAC: Authorizing `identity:get_user(user_id=2ad900f5d797453e94107cd46bf61c8a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:18.921089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:18.922768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 607/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:18 2022] GET /identity/v3/users/2ad900f5d797453e94107cd46bf61c8a => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:18.932032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:18.944632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:19.042690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bca6f0-c14d-46e9-b793-3bf6d31a2df5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:19.077919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:19.078553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:19.078837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:19.079109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:19.149209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:19.150598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:19.160346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:19.255563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 552/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:53:19.264654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:19.370423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0315cde-0687-4c76-8d8d-bc179b27052a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:19.405793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:19.406606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:19.406885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:19.407142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] PATH_INFO: `/v3/projects/ffcc7c72e12440e69aa054c52ea50968` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:19.408340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] RBAC: Authorizing `identity:get_project(project_id=ffcc7c72e12440e69aa054c52ea50968)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:19.409807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815f6d9-689b-48bc-bbad-29eaf5397a43 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:19.411530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 608/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:19 2022] GET /identity/v3/projects/ffcc7c72e12440e69aa054c52ea50968 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:53:19.418467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:19.535553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea827-6386-4008-9cd3-87bca9d19c3d None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:19.545223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:19.582469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:19.583468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:19.583903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:19.584325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:19.585717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=ffcc7c72e12440e69aa054c52ea50968, user.id=2ad900f5d797453e94107cd46bf61c8a) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:19.586190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:19.588508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b027ea8-8e94-453e-b54f-938a8445d9bb None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:19.596503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 553/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:53:19 2022] GET /identity/v3/role_assignments?user.id=2ad900f5d797453e94107cd46bf61c8a&scope.project.id=ffcc7c72e12440e69aa054c52ea50968&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:21.689232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ff27c-a4f3-401f-822e-cef11a6787b7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:21.690133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ff27c-a4f3-401f-822e-cef11a6787b7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:21.690133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ff27c-a4f3-401f-822e-cef11a6787b7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:21.692336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 609/1162] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:21.698833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:21.699294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:21.699387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:21.723681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:21.724898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:21.799612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 554/1163] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:21.823004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ff27c-a4f3-401f-822e-cef11a6787b7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:21.840096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:21.939068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ff27c-a4f3-401f-822e-cef11a6787b7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:21.975428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:21.975999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:21.976283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:21.976556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:22.051696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:22.053189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:22.068820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:22.153163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 610/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:53:22.160344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:22.269462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10975a1-ac71-43b5-bd07-30704ce10c27 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:22.271026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:22.312231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:22.313057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:22.313334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:22.313599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:22.320586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:22.322069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-7490cb69-5fc9-4e00-be4c-0bb8d782d5d0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:22.336634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:22.337855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:22.339043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 555/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:53:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:22.344514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:22.455781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc860b6-2798-4d72-bf00-88e443719d26 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:22.470610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:22.504761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:22.505460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:22.505743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:22.506006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:22.506735 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:22.507041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:22.508422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ab14-2c45-4604-bba8-e3fc02cd38f1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:22.515734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 611/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Jun 28 11:53:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:24.522933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d013-08a6-467d-9a61-11e4b44897ef None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:24.523943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d013-08a6-467d-9a61-11e4b44897ef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:24.523943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d013-08a6-467d-9a61-11e4b44897ef None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:24.526125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 556/1167] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:24.532590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e5037-8766-4268-8e36-2872bdbd8e88 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:24.533063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e5037-8766-4268-8e36-2872bdbd8e88 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:24.533195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e5037-8766-4268-8e36-2872bdbd8e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:24.560095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b98e5037-8766-4268-8e36-2872bdbd8e88 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:24.561374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b98e5037-8766-4268-8e36-2872bdbd8e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:24.632019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 612/1168] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:24.664334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d013-08a6-467d-9a61-11e4b44897ef None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:24.781052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d013-08a6-467d-9a61-11e4b44897ef None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:24.820197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:24.820861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:24.821193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:24.821466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:24.822208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:24.823606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9fa6-885b-4247-8dc9-d665ee98921b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:24.840491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 557/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Tue Jun 28 11:53:24 2022] POST /identity/v3/domains => generated 267 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:53:26.891208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a07901-80e2-4337-a8c7-0c02eed6b656 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:26.892090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a07901-80e2-4337-a8c7-0c02eed6b656 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:26.892090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a07901-80e2-4337-a8c7-0c02eed6b656 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:26.894431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 613/1170] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:26.900658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:26.901146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:26.901245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:26.920047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:26.921287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:26.938507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:26.996307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 558/1171] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:27.021564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a07901-80e2-4337-a8c7-0c02eed6b656 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:27.132003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a07901-80e2-4337-a8c7-0c02eed6b656 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:27.141835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:27.168229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:27.169287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:27.169628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:27.169909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] PATH_INFO: `/v3/domains/24528491e06e443ab00fced99d01ed06` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:27.180768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=24528491e06e443ab00fced99d01ed06)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:27.182239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb031e-35c1-4666-8ee2-a7cb328a6220 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:27.183872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 614/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Tue Jun 28 11:53:27 2022] GET /identity/v3/domains/24528491e06e443ab00fced99d01ed06 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:27.190000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:27.295666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920281c-433c-49a6-8df0-76ed7f08ba6e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:27.330919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:27.331571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:27.331850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:27.332117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:27.332834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:27.334318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a08e16-5319-4130-96ae-cfab2c907e14 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:27.365488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 559/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:53:27 2022] POST /identity/v3/projects => generated 361 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:53:29.485959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d84a2-aec4-4b39-b04b-b5ae08f2f979 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:29.486568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d84a2-aec4-4b39-b04b-b5ae08f2f979 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:29.486568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d84a2-aec4-4b39-b04b-b5ae08f2f979 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:29.487878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 615/1174] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:29.494317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:29.495039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:29.495234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:29.505592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:29.521499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:29.522729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:29.610290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 560/1175] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:29.646420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d84a2-aec4-4b39-b04b-b5ae08f2f979 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:29.754433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d84a2-aec4-4b39-b04b-b5ae08f2f979 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:29.794656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:29.795959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:29.796448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:29.796960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] PATH_INFO: `/v3/domains/24528491e06e443ab00fced99d01ed06` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:29.798373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=24528491e06e443ab00fced99d01ed06)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:29.800034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fbdcf-3ba8-4bc2-b781-5020502cf39b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:29.802233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 616/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Tue Jun 28 11:53:29 2022] GET /identity/v3/domains/24528491e06e443ab00fced99d01ed06 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:29.807801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:29.913655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95e57-1f85-4c40-89fb-b4a06799b91e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:29.951663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:29.952345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:29.952623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:29.952891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:29.953618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:29.955066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629cd27-2765-46db-91c9-2c7cf482b8b6 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:29.962362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:29.978784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 561/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:53:29 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:53:32.063261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c1f3-453b-4f70-b264-5833f2c61f32 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.064075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c1f3-453b-4f70-b264-5833f2c61f32 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.064075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c1f3-453b-4f70-b264-5833f2c61f32 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.066279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 617/1178] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:32.075046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.075494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.075630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.099517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:32.100712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:32.173445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 562/1179] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:32.197577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c1f3-453b-4f70-b264-5833f2c61f32 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:32.237555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:32.311567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c1f3-453b-4f70-b264-5833f2c61f32 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:32.346555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:32.347284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.347561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.347828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.359510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:32.366998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:32.368143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:53:32.369317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 618/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:32.375064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:32.486044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16300e-04e6-4b80-af6b-4e95784a922c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:32.508965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:32.531982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:32.532976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.533479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.533829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.534969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:32.535380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:32.537647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:32.548629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 563/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:32.557551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:32.668652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:32.675724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024736-0593-4c3d-b023-bc06960bd6e3 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:32.718963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:32.719849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.720129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.720401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] PATH_INFO: `/v3/users/7b58982bc7774883b16897c6979ee837` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.734275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] RBAC: Authorizing `identity:get_user(user_id=7b58982bc7774883b16897c6979ee837)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:32.735715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:32.737408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 619/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:32 2022] GET /identity/v3/users/7b58982bc7774883b16897c6979ee837 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:32.743758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:32.854318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0a039-f808-4931-a8f3-6d3b745f5ae8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:32.889825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:32.890463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:32.890736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:32.891005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:32.946550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:32.966659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:32.968133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:33.069661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 564/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:53:33.079430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:33.186544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47114f-d20b-4910-878a-7b9506db20eb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:33.222095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:33.222885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:33.223168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:33.223440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] PATH_INFO: `/v3/projects/5138e305101a4b62b1a1b4f1dc33885f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:33.224665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] RBAC: Authorizing `identity:get_project(project_id=5138e305101a4b62b1a1b4f1dc33885f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:33.226132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83b85a-c7d2-4c04-8ae3-8b7fa15b1572 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:33.227823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 620/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:33 2022] GET /identity/v3/projects/5138e305101a4b62b1a1b4f1dc33885f => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:33.234862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:33.316593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:33.336344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc33df-68d7-4e83-852d-7433e8b4ec71 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:33.372528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:33.373207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:33.373483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:33.373757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:33.374587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5138e305101a4b62b1a1b4f1dc33885f, user.id=7b58982bc7774883b16897c6979ee837) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:33.374898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:33.376327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afeb03-b326-4345-86f1-4693c377d95f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:33.382001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 565/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:53:33 2022] GET /identity/v3/role_assignments?user.id=7b58982bc7774883b16897c6979ee837&scope.project.id=5138e305101a4b62b1a1b4f1dc33885f&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:35.357787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2339-0312-4ee2-86eb-2f44029d0415 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:35.358363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2339-0312-4ee2-86eb-2f44029d0415 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:35.358705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2339-0312-4ee2-86eb-2f44029d0415 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:35.360572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 621/1186] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:35.367281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:35.368020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:35.368165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:35.391740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:35.392993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:35.469390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 566/1187] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:35.494861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2339-0312-4ee2-86eb-2f44029d0415 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:35.521902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:35.615083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2339-0312-4ee2-86eb-2f44029d0415 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:35.655940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:35.656743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:35.657054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:35.658429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:35.676015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:35.683943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:35.686052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:53:35.687412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 622/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:35.692400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:35.792380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:35.822308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdada79-5563-4bd0-abe6-84d0c774ac2b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:35.866022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:35.868145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:35.868432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:35.869704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:35.870489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:35.871721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:35.873230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:35.882440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 567/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:35.888975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:35.990356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:36.011963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ef2db-6b03-463b-bd55-d418a14e5e58 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:36.054256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:36.055149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:36.055388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:36.055915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] PATH_INFO: `/v3/users/7b58982bc7774883b16897c6979ee837` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:36.058001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] RBAC: Authorizing `identity:get_user(user_id=7b58982bc7774883b16897c6979ee837)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:36.059465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:36.061654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 623/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:35 2022] GET /identity/v3/users/7b58982bc7774883b16897c6979ee837 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:36.068004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:36.189358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ae95-167a-4a9e-9c55-1e62c284b0fc None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:36.230754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:36.231906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:36.232199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:36.232501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] PATH_INFO: `/v3/projects/5138e305101a4b62b1a1b4f1dc33885f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:36.233787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] RBAC: Authorizing `identity:get_project(project_id=5138e305101a4b62b1a1b4f1dc33885f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:36.235245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a26f3-3385-4376-8863-2a7bedf67104 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:36.237033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 568/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:36 2022] GET /identity/v3/projects/5138e305101a4b62b1a1b4f1dc33885f => generated 361 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:36.243658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:36.366314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2127-9251-47bf-84ae-7b40bc13b74a None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:36.422654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:36.423560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:36.423839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:36.424114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] PATH_INFO: `/v3/projects/5138e305101a4b62b1a1b4f1dc33885f/users/7b58982bc7774883b16897c6979ee837/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:36.426406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5138e305101a4b62b1a1b4f1dc33885f, user_id=7b58982bc7774883b16897c6979ee837, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:36.427860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6f0cf-439c-4fb5-a04d-e5a5e7336ba6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:36.443905 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 624/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:53:36 2022] PUT /identity/v3/projects/5138e305101a4b62b1a1b4f1dc33885f/users/7b58982bc7774883b16897c6979ee837/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:53:38.468851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e7a23-f665-4c44-a8bd-02a7c1362b88 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:38.469796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e7a23-f665-4c44-a8bd-02a7c1362b88 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:38.469796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e7a23-f665-4c44-a8bd-02a7c1362b88 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:38.471966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 569/1193] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:53:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:53:38.477611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:38.478033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:38.478122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:38.497054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:53:38.498344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:53:38.501174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:38.571986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 625/1194] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:53:38.596537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e7a23-f665-4c44-a8bd-02a7c1362b88 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:38.610867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:38.731695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e7a23-f665-4c44-a8bd-02a7c1362b88 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:38.766787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:38.767577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:38.767870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:38.768101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:38.783792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:53:38.791467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:53:38.792528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:53:38.793645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 570/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:53:38.799534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:38.912770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a7c94-5c6d-4db6-b348-8355e4826044 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:38.949753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:38.950456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:38.950799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:38.951088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:38.951826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:38.952130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:38.953546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:38.962362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 626/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:53:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:38.970069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:39.037998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:39.076710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d95f-ead8-411f-b629-8db60f0e78f6 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:39.115552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:39.116419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:39.116700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:39.117004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] PATH_INFO: `/v3/users/7b58982bc7774883b16897c6979ee837` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:39.118331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] RBAC: Authorizing `identity:get_user(user_id=7b58982bc7774883b16897c6979ee837)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:39.119777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:39.121436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 571/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Jun 28 11:53:38 2022] GET /identity/v3/users/7b58982bc7774883b16897c6979ee837 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:39.127976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:39.131034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:39.241137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45cec4-4414-4222-9db9-e1b8fdb80e53 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:39.279300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:39.279941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:39.280225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:39.280496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:39.352692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:39.354464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:39.356492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:39.470816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 627/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:53:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:53:39.477312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:39.584779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8dac8-b59f-4479-a2b2-251462dfe3ee None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:39.621020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:39.621842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:39.622192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:39.622518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] PATH_INFO: `/v3/projects/5138e305101a4b62b1a1b4f1dc33885f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:39.623785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] RBAC: Authorizing `identity:get_project(project_id=5138e305101a4b62b1a1b4f1dc33885f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:39.625268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fcd11-fe8d-4cd9-ac2d-5f359b813885 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:39.626965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 572/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Jun 28 11:53:39 2022] GET /identity/v3/projects/5138e305101a4b62b1a1b4f1dc33885f => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:39.633372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:53:39.746474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e35fb-41ad-431b-9808-3cb1bf53c98c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:53:39.748590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:39.785428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:53:39.786092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:53:39.786370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:53:39.786642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:53:39.787487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5138e305101a4b62b1a1b4f1dc33885f, user.id=7b58982bc7774883b16897c6979ee837) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:53:39.787813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:53:39.789284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8666b-5b69-4046-b699-589ff043aecb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:53:39.795458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 628/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:53:39 2022] GET /identity/v3/role_assignments?user.id=7b58982bc7774883b16897c6979ee837&scope.project.id=5138e305101a4b62b1a1b4f1dc33885f&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:53:49.615911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:53:49.792962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=17 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:36.183316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512228a-6fa1-4414-8dd7-51af08b2dbc8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:36.183867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512228a-6fa1-4414-8dd7-51af08b2dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:36.183867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512228a-6fa1-4414-8dd7-51af08b2dbc8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:36.185817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 573/1201] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:54:36.192100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:36.193157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:36.193157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:36.210898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:36.212083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:36.316650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 629/1202] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:54:36.346555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512228a-6fa1-4414-8dd7-51af08b2dbc8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:36.488746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512228a-6fa1-4414-8dd7-51af08b2dbc8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:36.528756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:36.529786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:36.530411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:36.530689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:36.544372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:36.546054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:36.565958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:36.567094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:36.568054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:36.568054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:36.568054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:36.568054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:36.568054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 574/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:36.573599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:36.706911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f404-0dc1-45a9-bf36-4e18e0c15b01 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:36.744021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:36.750816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:36.751542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:36.752169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:36.752525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:36.753937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:36.753937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:36.758617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a51b3-4064-4937-8bb9-623d9c752d55 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:36.780577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 630/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:36.782332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:36.853662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:36.912635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1636116-c025-49e1-8244-20f329c863ba None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:36.955298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:36.955874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:36.956166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:36.956524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:36.957412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:36.959104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ed298-e2e0-41db-bd47-0c8713c01fe3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:36.981757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 575/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Jun 28 11:54:36 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:54:39.327772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b4a0-1f4c-4be0-a19d-ce3916fdfcac None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:39.328358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b4a0-1f4c-4be0-a19d-ce3916fdfcac None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:39.328456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b4a0-1f4c-4be0-a19d-ce3916fdfcac None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:39.330014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 631/1206] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:54:39.335669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:39.336125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:39.336244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:39.355250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:39.357689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:39.444946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 576/1207] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:54:39.464264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203b4a0-1f4c-4be0-a19d-ce3916fdfcac None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:39.588143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203b4a0-1f4c-4be0-a19d-ce3916fdfcac None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:39.628087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:39.629199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:39.629530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:39.629796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:39.651323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:54:39.661005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:39.661984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:39.662890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 632/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:39.668609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:39.714768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:39.792017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1276709-04c9-4da2-ab9b-82ab6bb614d0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:39.827110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:39.827774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:39.828053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:39.828331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:39.829106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:39.829429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:39.830789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:39.839327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 577/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:54:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:54:39.845283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:39.907904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:39.974027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caab424-2d5c-4813-b2e9-1bf5e26d6379 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:40.019326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:40.021331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:40.021331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:40.021331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:40.035564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:40.037534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:40.060143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:40.061367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:40.062333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:40.062333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:40.062333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:40.062333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:40.062685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 633/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:40.067926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:40.190586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:40.195310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3d879-4e32-4a65-b1f7-9db2dfeff910 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:40.246769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:40.248017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:40.248506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:40.249020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:40.250359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:40.250895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:40.253349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:40.270791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 578/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:40.278120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:40.403180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0052f4-b00d-4ff9-b502-e647a52e7ec7 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:40.405129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:40.443473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:40.443473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:40.443473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:40.443473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:40.452407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:40.454605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:54:40.469726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:40.470753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:40.471889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:40.475618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 634/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:40.477409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:40.599543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bf8bc-9878-449b-94e7-1fece8b8318e None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:40.636648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:40.637414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:40.637796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:40.638727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:40.639642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:40.639949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:40.641430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:40.659946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 579/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:54:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:40.666071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:40.786682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3556307-5604-4b96-889c-9d59f2583e01 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:40.827327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:40.827327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:40.827734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:40.828197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:40.867723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:40.914212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:40.916056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.043739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 635/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:54:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:54:41.050252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:41.071645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:41.161541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8f0e2-d2a3-4338-bd84-f845c987116f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:41.196632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:41.197464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:41.197746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:41.198027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:41.209073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:41.210569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:41.229926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:41.231117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:41.236642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:41.236642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:41.236642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:41.236642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:41.236642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 580/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:41.237180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:41.309625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:41.354630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8e487-0358-466c-9e7c-1c346ed87852 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:41.390694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:41.391384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:41.391678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:41.392027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:41.392882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:41.393221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:41.394741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.407539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 636/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:41.413568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:41.502122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:41.546090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaae4e5-90d7-45ff-a01e-f1938cdbd26b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:41.595562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:41.596300 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:41.596609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:41.596955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:41.605764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:41.607676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:54:41.622482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:41.623523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:41.629398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 581/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:54:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:41.631564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:41.746834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba98b9-ca77-4d05-9c4a-f265939c012c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:41.773162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:41.789196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:41.789970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:41.790253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:41.790653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:41.791440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:41.791773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:41.793270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55522e8-b314-4513-8a29-9b5872d504c1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.806078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 637/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:41.812307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:41.925005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aaf5d-77fe-4b13-bb1a-d17f4f249b86 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:41.958325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:41.967861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:41.968734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:41.969050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:41.969418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:41.970425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=9bb4ac455d464617840790998ceea9c1) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:41.970813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:41.972278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f09db1-c79f-4c02-bed9-afec4d585d8a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:41.979123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 582/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:54:41 2022] GET /identity/v3/role_assignments?user.id=9bb4ac455d464617840790998ceea9c1&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:54:44.128860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aabe-b792-42cb-9d0c-23906a41b542 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.129444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aabe-b792-42cb-9d0c-23906a41b542 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.129444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aabe-b792-42cb-9d0c-23906a41b542 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.130743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 638/1220] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:54:44.136119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.136556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.136667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.155444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:44.156798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:44.235169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 583/1221] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:54:44.259465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835aabe-b792-42cb-9d0c-23906a41b542 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:44.371257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835aabe-b792-42cb-9d0c-23906a41b542 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:44.409641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:44.410625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.411006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.411457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.423982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:54:44.433068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:44.434240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:44.435908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 639/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:54:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:44.441998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:44.554488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b879dfc-acd4-4f9f-8a61-0fb855f43e5e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:44.579134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:44.590331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:44.591064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.591345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.591617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.592459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:44.592765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:44.594190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:44.602851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 584/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:54:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:44.609075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:44.656383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:44.722114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8009d-a00e-496b-9427-1082ef4d3050 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:44.775710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:44.776527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.776808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.777115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.790641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:44.792584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:44.813080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:44.814264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:44.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:44.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:44.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:44.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:44.815477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 640/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:44.820241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:44.926294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4590c-cede-44ad-845f-ec4b53661e23 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:44.963274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:44.963998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:44.964455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:44.964748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:44.965705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:44.966184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:44.967910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:44.984534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 585/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:44.990784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:45.104510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:45.111898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39901d0f-d997-4953-9fb6-e541208425fb None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:45.147519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:45.148491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:45.148801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:45.149175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:45.157606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:45.159076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:54:45.173656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:45.174823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:45.176093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 641/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:45.180871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:45.231945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:45.291350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394bbb-7dc8-4bc8-af5d-5513a1e876b4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:45.328553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:45.329397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:45.329740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:45.330104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:45.331310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:45.331677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:45.333489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:45.354630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 586/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:54:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:54:45.360710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:45.478581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753f26b-d685-4a82-8d3e-bc48a6c8c846 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:45.518406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:45.519366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:45.519697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:45.519981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:45.532063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:45.533644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:45.553210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:45.554593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:45.555612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:45.555612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:45.555612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:45.555612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:45.555612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 642/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:45.561271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:45.691522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80890c38-2015-4dff-8fdb-6948510a1721 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:45.694178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:45.731703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:45.732452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:45.732722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:45.733037 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:45.733886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:45.734185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:45.735824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:45.750464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 587/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:45.756655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:45.767138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:45.872650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c32222-d9e8-4278-a048-a8fc858a7523 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:45.912208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:45.913052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:45.913337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:45.913612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:45.920804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:45.922582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:54:45.938903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:45.940192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:45.941607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 643/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:45.946736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:46.065554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca162a3-bbd9-42d7-84cb-a3713552b4d5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:46.103197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:46.103920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:46.104244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:46.104680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:46.105718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:46.106077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:46.107738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b62a29-b097-4d08-b511-1641da8a2a3f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:46.120562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 588/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:46.127307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:46.195549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:46.234582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607574e-0634-4a0d-bd75-392540aee3f5 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:46.272923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:46.273830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:46.274113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:46.274388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/9bb4ac455d464617840790998ceea9c1/roles/bdc5c8dff27f406c9abe915ac2e783a6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:46.291530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=9bb4ac455d464617840790998ceea9c1, role_id=bdc5c8dff27f406c9abe915ac2e783a6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:46.293174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055f325-cd07-4a4a-8da9-f25c1dacee90 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:46.308674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 644/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:54:46 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/9bb4ac455d464617840790998ceea9c1/roles/bdc5c8dff27f406c9abe915ac2e783a6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:54:48.517899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fbb9e-a9d9-48bf-9fda-f518f8141f6e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:48.518515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fbb9e-a9d9-48bf-9fda-f518f8141f6e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:48.518515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fbb9e-a9d9-48bf-9fda-f518f8141f6e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:48.520141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 589/1233] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:54:48.525438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477f38-330f-432a-830d-256cb139858a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:48.525901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477f38-330f-432a-830d-256cb139858a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:48.525995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477f38-330f-432a-830d-256cb139858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:48.543649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8c477f38-330f-432a-830d-256cb139858a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:48.544965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8c477f38-330f-432a-830d-256cb139858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:48.620932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 645/1234] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 28 11:54:48.641219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fbb9e-a9d9-48bf-9fda-f518f8141f6e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:48.666770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:48.755929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fbb9e-a9d9-48bf-9fda-f518f8141f6e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:48.796078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:48.796991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:48.797308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:48.797604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:48.808491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:54:48.816923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:48.818375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:48.819425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 590/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Jun 28 11:54:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:48.824820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477f38-330f-432a-830d-256cb139858a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:48.885271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:48.947285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c477f38-330f-432a-830d-256cb139858a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:48.991410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:48.992143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:48.992475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:48.992809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:48.993653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] RBAC: Adding query filter params (name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:48.993990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:48.995391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:49.004541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 646/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Jun 28 11:54:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:49.010774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:49.110895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:49.129844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c033c-6d8e-4370-bdb4-7dbc4dfba801 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:49.172839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:49.173694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:49.173978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:49.174255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:49.188714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:49.190232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:49.209986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:49.211420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:49.212516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:49.212516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:49.212516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:49.212516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:49.212516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 591/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:54:49.219323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:49.332851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce6dc-f173-44de-818b-545c451a0927 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:49.343439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:49.369880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:49.370837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:49.371166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:49.371478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:49.372308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:49.372637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:49.374050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:49.385732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 647/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:49.392120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:49.509336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c854af-6832-40d7-8d59-d963f2461c6b None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:49.543626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:49.544947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:49.544947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:49.545150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:49.548126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:49.554023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:49.555504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:54:49.570046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:49.571251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:49.572360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 592/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:49.577702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:49.697779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6178-18f2-4ec3-9173-073a47095e5b None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:49.735559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:49.736348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:49.736668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:49.737017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:49.737924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:49.738222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:49.739664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:49.759316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 648/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:54:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:49.766813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:49.888481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe03e5b-107c-4b40-ac17-dcace8752357 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:49.925380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:49.926342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:49.926625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:49.926899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:49.998308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:50.008445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:50.011242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:50.132223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 593/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:54:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:54:50.138700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:50.171964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:50.261570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391115e-b28a-4d93-9b79-539127adb42f None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:50.303544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:50.304423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:50.304760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:50.305130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:50.316640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:50.318389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:50.338076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:50.339221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:50.340802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:50.340802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:50.340802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:50.340802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:50.340802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 649/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:50.346631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:50.463320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:50.470904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8d579-5106-4af4-93bf-0a493e6f4e53 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:50.508420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:50.509118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:50.509402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:50.509671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:50.510391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:50.510722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:50.512121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:50.523824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 594/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:50.530015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:50.613274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:50.638914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd9cf-82ed-4b6e-9078-1d766f05ca4c None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:50.680925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:50.681882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:50.682199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:50.682540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:50.691023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:50.692822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:54:50.708166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:50.709220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:50.710225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 650/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:50.716092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:50.825674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b174e-c685-4051-a31e-a6a9400ad82c None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:50.868092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:50.868769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:50.869085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:50.869355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:50.870203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:50.870501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:50.872231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a514233-fb6e-47d2-be6e-12df50c5e160 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:50.886909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 595/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:50.893950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:51.005450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4795dc2-79c2-4d86-b5a2-3eacf9784371 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:51.042979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:51.043569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:51.043854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:51.044134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:51.045151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] RBAC: Adding query filter params (role.id=bdc5c8dff27f406c9abe915ac2e783a6, scope.project.id=5709e926be584e859153090dd261a990, user.id=9bb4ac455d464617840790998ceea9c1) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:51.045488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:51.047185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e9fa-abe4-44ee-af3d-1bf3063f5fcb None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:51.062456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 651/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:54:50 2022] GET /identity/v3/role_assignments?user.id=9bb4ac455d464617840790998ceea9c1&scope.project.id=5709e926be584e859153090dd261a990&role.id=bdc5c8dff27f406c9abe915ac2e783a6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:53.216745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ec14-e546-43ac-9678-34d52f0d13d2 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:53.217367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ec14-e546-43ac-9678-34d52f0d13d2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:53.217367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4ec14-e546-43ac-9678-34d52f0d13d2 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:53.218706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 596/1247] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:54:53.226300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:53.226300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:53.226300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:53.230871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:53.244524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:53.246275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:53.319828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 652/1248] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:54:53.344894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4ec14-e546-43ac-9678-34d52f0d13d2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:53.366938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:53.477910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4ec14-e546-43ac-9678-34d52f0d13d2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:53.531587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:53.532546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:53.532857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:53.533239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:53.545895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:54:53.556617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:53.557904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:53.558938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 597/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:53.567150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:53.694550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241195a-d7e4-4988-a42b-ea6979590d6d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:53.726366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:53.733546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:53.734175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:53.734492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:53.734839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:53.735578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:53.735885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:53.737438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:53.747334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 653/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:53.753720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:53.843914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:53.870996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b7c-7926-4c5e-b480-e4793209a358 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:53.911427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:53.912749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:53.913256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:53.913712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:53.934979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:53.936500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:53.959167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:53.961189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:53.963212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:53.963212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:53.963212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:53.963212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:53.963212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 598/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:53.969566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:54.083781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0bedc-c0bc-476d-85b2-cfff31ec45d4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:54.128083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:54.129155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:54.129489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:54.129918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:54.130883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:54.131296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:54.133104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:54.152175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 654/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:54.160473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:54.276442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef76c-b739-4b5d-97a7-316c3f9412f3 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:54.310081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:54.320104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:54.321154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:54.321472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:54.321802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:54.330235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:54.331830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:54:54.347456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:54.348487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:54.349506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 599/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:54.355396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:54.416064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:54.468761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3dfbd-21b8-471d-a6d4-3f9102ed7725 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:54.510352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:54.511141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:54.511423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:54.511788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:54.512842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:54.513249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:54.514853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:54.534360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 655/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:54:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:54.540619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:54.655415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d67693-326b-41d5-b972-6b7f4a60f709 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:54.690218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:54.691126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:54.691446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:54.691883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:54.756065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:54.777594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:54.781657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:54.898651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 600/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:54:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 28 11:54:54.905284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:55.021239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9babc-36af-486a-b024-27e9c6cc3ca4 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:55.038225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:55.062766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:55.063588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:55.063854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:55.064130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:55.077655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:55.079165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:55.100733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:55.102416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:55.104346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:55.104346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:55.104346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:55.104346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:55.104346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 656/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:55.108724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:55.202208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:55.216929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba914d21-8d7a-4004-b9ac-eb6ff1ea6295 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:55.254736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:55.255473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:55.255813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:55.256116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:55.257094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:55.257420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:55.258982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:55.271494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 601/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:55.277800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:55.394526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a35d4-14e2-4ea6-b34d-a84a312f3d21 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:55.432779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:55.435400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:55.435762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:55.436107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:55.444438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:55.446289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:54:55.467054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:55.468305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:54:55.469429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 657/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:55.474766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:55.589324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c3e93-f51e-4ba8-9ded-6d09c9acac75 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:55.626942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:55.627729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:55.628012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:55.628284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:55.629208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:55.629514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:55.631150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8312cf-54bb-4d92-8992-9d6e077809d9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:55.644486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 602/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:55.650695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:55.679357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:55.761444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178739-b908-4803-9ab6-c4da3158c5d6 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:55.801093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:55.801818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:55.802125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:55.802419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:55.803331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=9bb4ac455d464617840790998ceea9c1) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:55.803669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:55.805262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a320e-890a-4cd4-b2d7-7f8055c5d8a2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:55.811649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 658/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:54:55 2022] GET /identity/v3/role_assignments?user.id=9bb4ac455d464617840790998ceea9c1&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:57.837223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c854f8-b201-47ef-b60c-b0fe4c9231f6 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:57.837827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c854f8-b201-47ef-b60c-b0fe4c9231f6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:57.837827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c854f8-b201-47ef-b60c-b0fe4c9231f6 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:57.839861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 603/1261] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:54:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:54:57.846187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:57.846930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:57.847080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:57.868994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:54:57.870136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:54:57.946710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 659/1262] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:54:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:54:57.972030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c854f8-b201-47ef-b60c-b0fe4c9231f6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:57.984432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:58.088812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c854f8-b201-47ef-b60c-b0fe4c9231f6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:58.124608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:58.125432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:58.125714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:58.125989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:58.136932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:54:58.144608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:58.145738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:58.146886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 604/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:54:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:58.152251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:58.173611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:58.254787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8ac7c-f6c5-496b-ba5b-cae3531ae039 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:58.291632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:58.292293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:58.292574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:58.292848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:58.293639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:58.294008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:58.295467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:58.303676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 660/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 11:54:58.309621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:58.402316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:58.442228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ea34-da24-487d-a9ba-0e1291cd2464 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:58.480767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:58.482066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:58.482369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:58.482677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:58.501100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:58.502606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:58.520154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:58.521185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:58.522248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:58.522248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:58.522248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:58.522248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:58.522248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 605/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:58.527491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:58.612732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:58.633265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebee99-3351-48bd-97e0-948baf969d36 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:58.668201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:58.668850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:58.669171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:58.669443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:58.670164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:58.670471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:58.671851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:58.683603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 661/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:58.691925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:58.795026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1ed21-89ff-43bf-8f04-cf0a1b1ca5c9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:58.814368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:58.828859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:58.829696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:58.829979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:58.830257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:58.838241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:58.840253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:54:58.861514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:58.862605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:58.863712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 606/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:58.868773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:58.998387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81cf09-c978-48f3-8cb1-f1ba9f8c5df9 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.034011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.034684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.034963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.035234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.036078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:59.036386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.037857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.046569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:59.055480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 662/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:54:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:59.063100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:59.165632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ece-f489-4109-9584-6fc2aecba239 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.201412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.202198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.202475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.202742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.213678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.215136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:54:59.233316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:59.234353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:59.235454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:54:59.235454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:54:59.235454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:54:59.235454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:59.235454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 607/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:59.240311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:59.350630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318291-cb85-4aa4-b6df-dc972e2d4951 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.386871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.388015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.388302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.388566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.389316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:59.389616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.390983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.402114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 663/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:59.408776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:59.451781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:59.524480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c40fbd-4cf8-4b19-9f12-465748cf649f None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.558629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.560374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.560657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.560949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.568277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.569771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:54:59.585310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:54:59.586327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:54:59.587626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 608/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:54:59.595398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:59.656146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:59.700499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4abf5-37d6-4e6b-abfb-07370e407461 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.738100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.738757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.739040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.739316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.740111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:54:59.740422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.741910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0fd75-7cf4-4065-9fe1-a04449d69bcc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.754290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 664/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:54:59.763393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:54:59.861303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:54:59.875360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712ebf-82e9-4bc3-badb-c113357c84a9 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:54:59.912250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:54:59.913104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:54:59.913381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:54:59.913653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] PATH_INFO: `/v3/projects/5709e926be584e859153090dd261a990/users/9bb4ac455d464617840790998ceea9c1/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:54:59.915766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5709e926be584e859153090dd261a990, user_id=9bb4ac455d464617840790998ceea9c1, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:54:59.917249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420470c-15a1-4d8d-90e5-92d7870669f9 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:54:59.933398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 609/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Jun 28 11:54:59 2022] PUT /identity/v3/projects/5709e926be584e859153090dd261a990/users/9bb4ac455d464617840790998ceea9c1/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 11:55:01.946961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea439-ea95-40a6-987e-c7ce717a152d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:01.947839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea439-ea95-40a6-987e-c7ce717a152d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:01.947839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea439-ea95-40a6-987e-c7ce717a152d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:01.950906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 665/1274] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:55:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:55:01.956375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:01.956817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:01.956950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:01.977338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:55:01.978539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:55:02.052206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 610/1275] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:55:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:55:02.076524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea439-ea95-40a6-987e-c7ce717a152d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:02.200983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea439-ea95-40a6-987e-c7ce717a152d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:02.218831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:02.248922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:02.249734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:02.250016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:02.250245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:02.261031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 28 11:55:02.268605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:02.269759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:02.270707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 666/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Jun 28 11:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:02.276087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:02.373019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:02.392576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530cb63-8360-4dbe-880b-8bfbd1caddd8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:02.428869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:02.429736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:02.430071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:02.430472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:02.431461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:02.431942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:02.433791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:02.446213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 611/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Jun 28 11:55:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:02.453658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:02.581293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf9-e079-4bce-bd22-e7ccc29fd0dd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:02.617537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:02.618328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:02.618611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:02.618881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:02.630956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:02.632356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:55:02.649748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:02.650792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:02.656883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:55:02.656883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:55:02.656883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:55:02.656883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:02.656883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 667/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:02.657404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:02.764288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaeba87-0cc3-4215-ae00-7f0a46605663 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:02.799207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:02.799870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:02.800146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:02.800412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:02.801214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:02.801522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:02.802870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:02.816454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 612/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:02.825588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:02.865489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:02.943129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b99c6f-ce5d-4803-bfcf-425c82143db1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:02.978992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:02.979764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:02.980043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:02.980337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:02.988162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:02.989664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 28 11:55:03.004190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:03.005322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:03.006620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 668/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:55:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:03.011963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:03.017802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:03.141970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677fb5b-c72d-4764-8481-d6418443a0c4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:03.196928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:03.197524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:03.197796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:03.198061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:03.199016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:03.199321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:03.201803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:03.220783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 613/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Jun 28 11:55:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:03.228223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:03.344943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:03.364137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94cc33-8535-4eaa-9d07-30717b4f29f2 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:03.404630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:03.405569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:03.405941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:03.406316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:03.487392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:03.488815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:03.585447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:03.597711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 669/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Jun 28 11:55:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 28 11:55:03.604115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:03.666506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:03.707873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a55a8f-9b1a-418d-80f5-dd3acacf7519 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:03.747976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:03.748711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:03.749016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:03.749289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:03.759479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:03.760945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 28 11:55:03.780153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:03.781168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:03.782287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 28 11:55:03.782287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 28 11:55:03.782287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 28 11:55:03.782287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:55:03.782287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 614/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 28 11:55:03.787269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:03.888673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528c48d-eaf2-4a18-a70f-f4d967baa023 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:03.923917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:03.924566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:03.924843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:03.925138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:03.925856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:03.926167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:03.927544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:03.938847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 670/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Jun 28 11:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:03.947843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:04.055536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97072a6f-c0b8-461d-a442-1285488234e8 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:04.069455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:04.091720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:04.092509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:04.092785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:04.093083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:04.100120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:04.101993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 28 11:55:04.116187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:04.117215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 28 11:55:04.118134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: ERROR keystone.server.flask.application Jun 28 11:55:04.121101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 615/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Jun 28 11:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:04.123500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:04.176457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:04.230461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551d594-9ffa-4ae7-879c-a4a4271ed4c1 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:04.266503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:04.267159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:04.267439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:04.267712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:04.268507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:04.268818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:04.270306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16668738-7c42-4174-bd95-7c5d7f954b62 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:04.283419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 671/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Jun 28 11:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:04.289734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:04.395078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719337-e519-44c3-91a3-d025676a59e5 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:04.443265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:04.443973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:04.444319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:04.444666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:04.445739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] RBAC: Adding query filter params (role.id=a034d6cd495d4d39acddae91f37019a3, scope.project.id=5709e926be584e859153090dd261a990, user.id=9bb4ac455d464617840790998ceea9c1) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:04.446090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:04.448015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3a24b-1053-4b47-b23a-19960af2f666 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:04.454824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 616/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Jun 28 11:55:04 2022] GET /identity/v3/role_assignments?user.id=9bb4ac455d464617840790998ceea9c1&scope.project.id=5709e926be584e859153090dd261a990&role.id=a034d6cd495d4d39acddae91f37019a3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:06.511506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e5983-f49c-42d5-8451-a186bbf5694d None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:06.512725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e5983-f49c-42d5-8451-a186bbf5694d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:06.512725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e5983-f49c-42d5-8451-a186bbf5694d None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:06.514681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 672/1288] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:55:06.520167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:06.520592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:06.520681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:06.538929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:55:06.540107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:55:06.570913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:06.611537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 617/1289] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:55:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:55:06.634255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e5983-f49c-42d5-8451-a186bbf5694d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:06.742888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e5983-f49c-42d5-8451-a186bbf5694d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:06.753546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:06.779146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:06.779951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:06.780226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:06.780496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:06.781185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:06.782567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:06.795523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:06.796706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:06.797648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 673/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:55:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:06.802793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:06.919004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546f02b-a5d3-45c1-b290-1620982aea57 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:06.957590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:06.958253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:06.958534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:06.958808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:06.959563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:06.959873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:06.961600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a724ae-45d5-4050-bb7f-55da3f532316 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:06.966751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 618/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Jun 28 11:55:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:06.973046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:07.078408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e2ae7-317a-48db-9cb8-c10a778820cd None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:07.114462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:07.115118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:07.115401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:07.115672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] PATH_INFO: `/v3/services` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:07.116376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] RBAC: Adding query filter params (type=placement) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:07.116682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:07.118090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb188c94-99e6-40d5-a713-3153d581ebcc None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:07.122776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 674/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Jun 28 11:55:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:09.169492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478071-fc2a-4174-b0a4-a79718cdd57a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:09.170538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478071-fc2a-4174-b0a4-a79718cdd57a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:09.170538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478071-fc2a-4174-b0a4-a79718cdd57a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:09.172416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 619/1293] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:55:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:55:09.178767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd315-9e07-4e26-8d01-37219a1cd8c8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:09.179487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd315-9e07-4e26-8d01-37219a1cd8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:09.179666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd315-9e07-4e26-8d01-37219a1cd8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:09.204690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5defd315-9e07-4e26-8d01-37219a1cd8c8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:55:09.205947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5defd315-9e07-4e26-8d01-37219a1cd8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:55:09.237410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:09.278203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 675/1294] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:55:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:55:09.302368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478071-fc2a-4174-b0a4-a79718cdd57a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:09.344669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:09.419628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478071-fc2a-4174-b0a4-a79718cdd57a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:09.460404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:09.461332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:09.461765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:09.462137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:09.463086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:09.464696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a52a2-4ced-4e6e-a1d8-d9ae28df171a None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:09.480940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 620/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Jun 28 11:55:09 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:55:11.534090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f92a9-df77-4fcc-99cb-ceffb9175c08 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:11.535005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f92a9-df77-4fcc-99cb-ceffb9175c08 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:11.535005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f92a9-df77-4fcc-99cb-ceffb9175c08 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:11.537274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 676/1296] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:55:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:55:11.543566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:11.544127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:11.544217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:11.562599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:55:11.563804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:55:11.647399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 621/1297] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:55:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:55:11.678989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f92a9-df77-4fcc-99cb-ceffb9175c08 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:11.794651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f92a9-df77-4fcc-99cb-ceffb9175c08 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:11.830624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:11.831420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:11.831701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:11.831978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:11.832629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:11.834044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:11.846240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:11.847283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:11.848312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 677/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:55:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:11.856924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:11.877990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:11.962448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580c8ca-42b0-4656-8364-c266a3fc1cb5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:11.997836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:11.998488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:11.998771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:11.999046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:11.999759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:12.000068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:12.001494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:12.006943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 622/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Jun 28 11:55:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:12.016681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:12.055631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:12.139748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57d9c-67ce-4dae-93f2-709348f55666 None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:12.176175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:12.176814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:12.177120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:12.177389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:12.178173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] RBAC: Adding query filter params (interface=public, service_id=a87ece46788f4074ba8981592a30345d, region_id=RegionOne) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:12.178477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:12.179851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191fb26-d9a4-4077-94ba-44640327bd32 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:12.186863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 678/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Jun 28 11:55:12 2022] GET /identity/v3/endpoints?service_id=a87ece46788f4074ba8981592a30345d&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:12.196272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:12.297175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:12.316821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed41cf-c956-4d6e-b483-ed87efb662e4 None admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:12.357206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:12.357872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:12.358203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:12.358614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:12.359510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:12.360951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb56a0e-2764-42c3-91a5-ff9ef162974b None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:12.368418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 623/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Jun 28 11:55:12 2022] GET /identity/v3/services => generated 2191 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:55:14.384443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9c5c6-a561-43cc-9054-f4e81983c3bb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:14.385398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9c5c6-a561-43cc-9054-f4e81983c3bb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:14.385398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9c5c6-a561-43cc-9054-f4e81983c3bb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:14.387615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 679/1302] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:55:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:55:14.393108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:14.393537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:14.393647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:14.411354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:55:14.412555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:55:14.483076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 624/1303] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:55:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 28 11:55:14.506316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9c5c6-a561-43cc-9054-f4e81983c3bb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:14.617089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:14.629420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9c5c6-a561-43cc-9054-f4e81983c3bb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:14.671476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:14.674792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:14.675073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:14.675346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:14.676002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:14.677410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 28 11:55:14.692741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 28 11:55:14.693931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.get_or_create( Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application with Lock( Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self._enter() Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application return self.creator() Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application created_value = creator( Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: ERROR keystone.server.flask.application Jun 28 11:55:14.695446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 680/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Jun 28 11:55:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 28 11:55:14.702789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:14.781020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:14.808003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c3a49-6bee-4be7-906a-cf14ba565b26 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:14.844821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:14.845519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:14.845802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:14.846070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] PATH_INFO: `/v3/services` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:14.846772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 11:55:14.847079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:14.848455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508a11-0705-43bd-b138-a942e9f4d747 None admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:14.854121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 625/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Jun 28 11:55:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 11:55:14.862856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:55:14.967879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3c60-868c-4b9b-b648-1f2c83c64e0e None admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:55:15.002632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:55:15.003283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:55:15.003558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:55:15.003831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:55:15.004562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:55:15.005971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd6611-072f-42aa-8510-7cc44115f3c7 None admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:55:15.034238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 681/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Jun 28 11:55:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 11:55:24.850578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=30 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:55:25.024976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:18.997541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c046-2877-4c04-b85d-bf0d49aa735b None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:18.998107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c046-2877-4c04-b85d-bf0d49aa735b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:18.998107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c046-2877-4c04-b85d-bf0d49aa735b None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:19.003666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 626/1307] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:19.005151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0194-aea0-4780-9f2a-e7a2f8b1574b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:19.005596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0194-aea0-4780-9f2a-e7a2f8b1574b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:19.005684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a0194-aea0-4780-9f2a-e7a2f8b1574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:19.109258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e28a0194-aea0-4780-9f2a-e7a2f8b1574b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:19.110426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e28a0194-aea0-4780-9f2a-e7a2f8b1574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:19.262827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 682/1308] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 11:56:19.300992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:19.301282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:19.301561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:19.303175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 627/1309] 10.222.0.55 () {54 vars in 964 bytes} [Tue Jun 28 11:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:19.310733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b5dd-0fc3-4dd7-8ecf-01d9dfc47714 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:19.312822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b5dd-0fc3-4dd7-8ecf-01d9dfc47714 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:19.313933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b5dd-0fc3-4dd7-8ecf-01d9dfc47714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:19.429787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-60c1b5dd-0fc3-4dd7-8ecf-01d9dfc47714 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:19.431162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-60c1b5dd-0fc3-4dd7-8ecf-01d9dfc47714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:19.453883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:19.513970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 683/1310] 10.222.0.55 () {58 vars in 1061 bytes} [Tue Jun 28 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:19.523582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:19.525546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:19.642560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36e95-6225-47e6-9a73-58eca38c29c0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:19.718634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:19.719289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:19.719564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:19.719828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:19.721493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:19.781355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:19.843944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:19.845439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852be8cb-e961-4e7a-81c9-9e2b19357641 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:19.866280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 628/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:22.228055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15868fa-a458-4738-9f37-9816ab780f21 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:22.228605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15868fa-a458-4738-9f37-9816ab780f21 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:22.228605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15868fa-a458-4738-9f37-9816ab780f21 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:22.230237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 684/1312] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:22.236457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b35f-ca44-4c41-bde3-ef86bf36791b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:22.237044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b35f-ca44-4c41-bde3-ef86bf36791b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:22.237044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b35f-ca44-4c41-bde3-ef86bf36791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:22.322816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8f99b35f-ca44-4c41-bde3-ef86bf36791b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:22.324054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8f99b35f-ca44-4c41-bde3-ef86bf36791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:22.431305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 629/1313] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:22.482524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:22.483149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:22.483428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:22.484821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 685/1314] 10.222.0.55 () {54 vars in 964 bytes} [Tue Jun 28 11:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:22.491559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7aeb35-982f-4487-86d3-324d804bf791 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:22.492052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7aeb35-982f-4487-86d3-324d804bf791 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:22.492470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7aeb35-982f-4487-86d3-324d804bf791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:22.505540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:22.588366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8e7aeb35-982f-4487-86d3-324d804bf791 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:22.589628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8e7aeb35-982f-4487-86d3-324d804bf791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:22.684747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 630/1315] 10.222.0.55 () {58 vars in 1061 bytes} [Tue Jun 28 11:56:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:22.694138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:22.695756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:22.809890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f26dea-f101-495c-8f75-ac43d6b701f8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:22.882852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:22.885839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:22.886599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:22.886890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:22.887174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:22.889440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:23.003016 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:23.004560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e3972-5940-468f-ac27-4007d4ee976f service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:23.023872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 686/1316] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:25.571441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:25.572042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:25.572042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:25.573419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 631/1317] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:25.579864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b743222-606e-4830-803e-484f50787d0b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:25.580569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b743222-606e-4830-803e-484f50787d0b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:25.580679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b743222-606e-4830-803e-484f50787d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:25.657801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8b743222-606e-4830-803e-484f50787d0b None None] MFA Rules not processed for user `1f6fbf1c7c8a43a0b25ed73d3762dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:25.659022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8b743222-606e-4830-803e-484f50787d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:25.716015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:25.768354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 687/1318] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:25.811334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:25.812950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:25.938714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bca623-011b-4603-a0ac-7c16e4af0e22 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:25.941735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:26.017966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:26.018617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:26.018896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:26.019166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:26.020799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:26.147904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:26.149409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e84d-810d-4ab2-a9fc-84a5bdfd717e service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:26.167884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 632/1319] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:28.819151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:28.819810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:28.819810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:28.821856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 688/1320] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:28.829400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f09f-7a70-45e5-b727-e088e070f3cc None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:28.829888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f09f-7a70-45e5-b727-e088e070f3cc None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:28.830077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f09f-7a70-45e5-b727-e088e070f3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:28.871212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:28.902332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5728f09f-7a70-45e5-b727-e088e070f3cc None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:28.904024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5728f09f-7a70-45e5-b727-e088e070f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:29.009614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 633/1321] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:29.053919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:29.055463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:29.170552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0459be-e6ae-420d-92e0-7d17dcc4c6b1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:29.247865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:29.248627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:29.249007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:29.249294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:29.250921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:29.258327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:29.365187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:29.366638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b69cb-6cb9-4352-97be-b84ca39f5dce service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:29.385879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 689/1322] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:32.106725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:32.107290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:32.107591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:32.110232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 634/1323] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:32.116024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502ee7f-b225-422a-9005-75f5d89dcd53 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:32.116472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502ee7f-b225-422a-9005-75f5d89dcd53 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:32.116733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502ee7f-b225-422a-9005-75f5d89dcd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:32.184477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4502ee7f-b225-422a-9005-75f5d89dcd53 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:32.185754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4502ee7f-b225-422a-9005-75f5d89dcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:32.248026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:32.290894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 690/1324] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:32.328649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:32.330328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:32.456272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905eaddb-eb32-4e17-97a8-ff09db358c22 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:32.469654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:32.537434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:32.538081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:32.538359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:32.538631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:32.540260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:32.652936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:32.654442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208a79-f0a3-4f8a-b2b2-e98fc00f1816 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:32.672995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 635/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:34.785149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:34.785637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:34.785756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:34.787156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 691/1326] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:34.793114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd607-a6de-4617-aaeb-ce3c1893a20a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:34.793572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd607-a6de-4617-aaeb-ce3c1893a20a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:34.793572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadd607-a6de-4617-aaeb-ce3c1893a20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:34.842703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:34.863131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3aadd607-a6de-4617-aaeb-ce3c1893a20a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:34.864339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3aadd607-a6de-4617-aaeb-ce3c1893a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:34.963932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 636/1327] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:35.008766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:35.010359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:35.118551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b3407-bd66-42c1-8f14-dabfefcf7017 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:35.193852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:35.194513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:35.194793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:35.195061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:35.196711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:35.215663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:35.317651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:35.319118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67faf16f-fae8-44c9-b42f-007b6ef12f2f service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:35.338316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 692/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:37.441179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:37.441773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:37.441773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:37.443802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 637/1329] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:37.451277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295177-6151-4bf2-9ba3-fa414009811a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:37.451800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295177-6151-4bf2-9ba3-fa414009811a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:37.451994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295177-6151-4bf2-9ba3-fa414009811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:37.517765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-10295177-6151-4bf2-9ba3-fa414009811a None None] MFA Rules not processed for user `1f6fbf1c7c8a43a0b25ed73d3762dff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:37.518985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-10295177-6151-4bf2-9ba3-fa414009811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:37.585444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:37.620251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 693/1330] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:37.658719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:37.660291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:37.776831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38382dde-3350-4af4-9af2-527d4ce87f55 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:37.799161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:37.854788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:37.855349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:37.855631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:37.855906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:37.857579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:37.973263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:37.975099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a9d7b-6b44-4dec-a9c5-d6fea9cb6ee4 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:37.994266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 638/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:40.518217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:40.518794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:40.518794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:40.520157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 694/1332] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:40.525766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267423-f993-4d86-a9e1-802ee0f56201 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:40.526233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267423-f993-4d86-a9e1-802ee0f56201 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:40.526328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267423-f993-4d86-a9e1-802ee0f56201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:40.594407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:40.597587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6b267423-f993-4d86-a9e1-802ee0f56201 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:40.599924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6b267423-f993-4d86-a9e1-802ee0f56201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:40.702915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 639/1333] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:40.739459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:40.741169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:40.850045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615d2a5-5fdc-47bb-ad93-25ee46c5de18 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:40.926684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:40.927337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:40.927620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:40.927897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:40.929579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:40.959233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:41.047061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:41.048415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a5ed-896e-42a1-93d4-18dd4f0a16a2 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:41.067044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 695/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:43.174675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:43.175274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:43.175274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:43.176625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 640/1335] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:43.183146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ddc0-18ce-4e8d-b996-d74a5a8cfd50 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:43.183915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ddc0-18ce-4e8d-b996-d74a5a8cfd50 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:43.184079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3ddc0-18ce-4e8d-b996-d74a5a8cfd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:43.251638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8bb3ddc0-18ce-4e8d-b996-d74a5a8cfd50 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:43.252846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8bb3ddc0-18ce-4e8d-b996-d74a5a8cfd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:43.345004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:43.378799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 696/1336] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:43.422524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:43.424073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:43.543178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbccfc-24d8-4d05-9973-8b913d01cb43 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:43.575780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:43.625819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:43.626466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:43.626744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:43.627015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:43.628632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:43.759857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:43.761358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b5120-e80b-4c73-9c93-8cd115bff385 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:43.782986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 641/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:46.175984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:46.176520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:46.176520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:46.178654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 697/1338] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:46.187316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092dcf-4df1-451e-b84f-6e28e10fe37e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:46.187706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092dcf-4df1-451e-b84f-6e28e10fe37e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:46.187860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7092dcf-4df1-451e-b84f-6e28e10fe37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:46.259442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b7092dcf-4df1-451e-b84f-6e28e10fe37e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:46.260646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b7092dcf-4df1-451e-b84f-6e28e10fe37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:46.269418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:46.384747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 642/1339] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:46.425368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:46.427942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:46.562280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e27ed7-75f7-4f3a-82b3-6890ef420ed0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:46.640001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:46.640810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:46.641129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:46.641403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:46.643040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:46.682464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:46.769384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:46.770847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209909b-b857-4fec-bf3b-37148e3ded4e service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:46.791340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 698/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:51.048354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:51.049000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:51.049000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:51.050355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 643/1341] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:51.055949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a54ae-fb9e-4afe-a8cd-8da65b084492 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:51.056355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a54ae-fb9e-4afe-a8cd-8da65b084492 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:51.056719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a54ae-fb9e-4afe-a8cd-8da65b084492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:51.132581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f01a54ae-fb9e-4afe-a8cd-8da65b084492 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:51.133833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f01a54ae-fb9e-4afe-a8cd-8da65b084492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:51.225787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:51.239978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 699/1342] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:51.279325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:51.281744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:51.380414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce59b9-d159-4e17-826c-e0b3033212f4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:51.384303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:51.384970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:51.385250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:51.385521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:51.394950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:51.396448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312151a-ed8d-4222-b1e2-b9077cfbb5a2 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:51.406091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 644/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:54.090521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:54.091012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:54.091106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:54.093133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 700/1344] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:54.098486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f6e7-19ab-4e3f-947c-76ec7aef5154 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:54.098961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f6e7-19ab-4e3f-947c-76ec7aef5154 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:54.099524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137f6e7-19ab-4e3f-947c-76ec7aef5154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:54.130813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a137f6e7-19ab-4e3f-947c-76ec7aef5154 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:54.132061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a137f6e7-19ab-4e3f-947c-76ec7aef5154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:54.166412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:56:54.222069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 645/1345] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:54.261599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:54.263450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:54.278113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73042a-3775-40e3-92c9-4c25ddce14be None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:54.282441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:54.283110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:54.283396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:54.283680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:54.293531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:54.294993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65096343-1134-4d59-9edf-a612bf863c64 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:54.304948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 701/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:56:58.648643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333cb7-0673-49e9-8463-59faab97e5c4 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:58.649292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333cb7-0673-49e9-8463-59faab97e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:58.649292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333cb7-0673-49e9-8463-59faab97e5c4 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:58.650770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 646/1347] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:56:58.656078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faaf51-1deb-45fc-81cf-fa4ea3373890 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:58.656521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faaf51-1deb-45fc-81cf-fa4ea3373890 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:58.656611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faaf51-1deb-45fc-81cf-fa4ea3373890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:58.674780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a3faaf51-1deb-45fc-81cf-fa4ea3373890 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:56:58.675997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a3faaf51-1deb-45fc-81cf-fa4ea3373890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:56:58.684877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 702/1348] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:56:58.724566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333cb7-0673-49e9-8463-59faab97e5c4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:56:58.736525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333cb7-0673-49e9-8463-59faab97e5c4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:56:58.740651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:56:58.741359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:56:58.741703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:56:58.741985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:56:58.751921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:56:58.753425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1603f4-dae2-4d6e-a9a2-46bcb649339b service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:56:58.762889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 647/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:03.723596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0998a8-1cbf-4067-9266-445d2b6224d0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:03.724805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0998a8-1cbf-4067-9266-445d2b6224d0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:03.725286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0998a8-1cbf-4067-9266-445d2b6224d0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:03.726946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 703/1350] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 11:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:03.735825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cdff7-bda3-4228-9b2c-77b7dbe18f02 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:03.736236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cdff7-bda3-4228-9b2c-77b7dbe18f02 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:03.736599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cdff7-bda3-4228-9b2c-77b7dbe18f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:03.829436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a93cdff7-bda3-4228-9b2c-77b7dbe18f02 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:03.831008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a93cdff7-bda3-4228-9b2c-77b7dbe18f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:03.962879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 648/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 11:57:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:08.362886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aee163-547b-4d6d-afd2-fd7ec7f3ac9e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:08.363438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aee163-547b-4d6d-afd2-fd7ec7f3ac9e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:08.363438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aee163-547b-4d6d-afd2-fd7ec7f3ac9e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:08.365008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 704/1352] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 11:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:57:08.370087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08275-fa17-4218-9440-aa3d5a54da26 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:08.370893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08275-fa17-4218-9440-aa3d5a54da26 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:08.371382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a08275-fa17-4218-9440-aa3d5a54da26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:08.395327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f9a08275-fa17-4218-9440-aa3d5a54da26 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:08.396807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f9a08275-fa17-4218-9440-aa3d5a54da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:08.407717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 649/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:08.669383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=13 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:09.655426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f25937-6dff-42a0-9771-ea39acf88b12 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:09.656095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f25937-6dff-42a0-9771-ea39acf88b12 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:09.656095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f25937-6dff-42a0-9771-ea39acf88b12 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:09.657594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 705/1354] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 11:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:09.662687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593105-aecd-47d5-95d8-d05803048ad6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:09.663159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593105-aecd-47d5-95d8-d05803048ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:09.663257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1593105-aecd-47d5-95d8-d05803048ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:09.682268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f1593105-aecd-47d5-95d8-d05803048ad6 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:09.683482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f1593105-aecd-47d5-95d8-d05803048ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:09.692929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 650/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 11:57:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:13.309902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b646-b1e7-44db-8d90-94c53852c22e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:13.310777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b646-b1e7-44db-8d90-94c53852c22e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:13.311142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b646-b1e7-44db-8d90-94c53852c22e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:13.317896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 706/1356] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:13.318628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6006d0-5b17-4a99-a2dc-b6007c67cbe5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:13.319476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6006d0-5b17-4a99-a2dc-b6007c67cbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:13.319950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6006d0-5b17-4a99-a2dc-b6007c67cbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:13.344217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9e6006d0-5b17-4a99-a2dc-b6007c67cbe5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:13.345864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9e6006d0-5b17-4a99-a2dc-b6007c67cbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:13.354931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 651/1357] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:13.412307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc424a-aced-4a03-a509-b35d7bf53bb1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:13.412594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc424a-aced-4a03-a509-b35d7bf53bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:13.413088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc424a-aced-4a03-a509-b35d7bf53bb1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:13.414477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 707/1358] 10.222.0.55 () {54 vars in 961 bytes} [Tue Jun 28 11:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:13.423432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf286fe-4133-4535-9cce-b8215bc3e07e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:13.423988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf286fe-4133-4535-9cce-b8215bc3e07e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:13.424707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf286fe-4133-4535-9cce-b8215bc3e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:13.475415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:13.484623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0bf286fe-4133-4535-9cce-b8215bc3e07e None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:13.485930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0bf286fe-4133-4535-9cce-b8215bc3e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:13.570209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 652/1359] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Jun 28 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:13.579262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc424a-aced-4a03-a509-b35d7bf53bb1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:13.594548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc424a-aced-4a03-a509-b35d7bf53bb1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:13.601200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:13.602188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:13.602754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:13.603210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:13.613764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:13.615601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6204-c609-4352-b001-53b0a5548138 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:13.626096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 708/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:15.872616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5b34-fe59-4316-aa7c-bc93f8020d08 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:15.873169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5b34-fe59-4316-aa7c-bc93f8020d08 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:15.873169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a5b34-fe59-4316-aa7c-bc93f8020d08 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:15.874973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 653/1361] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:15.880207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cf1b-4aa5-4088-9604-afc788538165 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:15.880649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cf1b-4aa5-4088-9604-afc788538165 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:15.880748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5cf1b-4aa5-4088-9604-afc788538165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:15.899724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-43c5cf1b-4aa5-4088-9604-afc788538165 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:15.900948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-43c5cf1b-4aa5-4088-9604-afc788538165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:15.913023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 709/1362] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:15.954601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a5b34-fe59-4316-aa7c-bc93f8020d08 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:15.969643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a5b34-fe59-4316-aa7c-bc93f8020d08 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:15.975862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:15.976648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:15.976989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:15.977353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:15.986894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:15.988813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b5c06-3069-4af9-b6bb-e0312b2022bd service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:16.006609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 654/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:18.360810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6573-54b4-4a66-8252-132da2260462 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:18.361453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6573-54b4-4a66-8252-132da2260462 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:18.368243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6573-54b4-4a66-8252-132da2260462 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:18.370748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 710/1364] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:18.376531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b773-49b4-4357-bb9a-ce48e6fbb31d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:18.377062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b773-49b4-4357-bb9a-ce48e6fbb31d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:18.377154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249b773-49b4-4357-bb9a-ce48e6fbb31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:18.398759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e249b773-49b4-4357-bb9a-ce48e6fbb31d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:18.399981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e249b773-49b4-4357-bb9a-ce48e6fbb31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:18.408095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 655/1365] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:18.454845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977cd5-7660-4ca4-a0f4-219868543be1 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:18.455258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977cd5-7660-4ca4-a0f4-219868543be1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:18.455710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d977cd5-7660-4ca4-a0f4-219868543be1 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:18.457131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 711/1366] 10.222.0.55 () {54 vars in 961 bytes} [Tue Jun 28 11:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:18.462657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91494af5-1c3b-4a19-b367-8aaced55fb1d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:18.463057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91494af5-1c3b-4a19-b367-8aaced55fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:18.463432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91494af5-1c3b-4a19-b367-8aaced55fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:18.483281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-91494af5-1c3b-4a19-b367-8aaced55fb1d None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:18.484517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-91494af5-1c3b-4a19-b367-8aaced55fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:18.494405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 656/1367] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Jun 28 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 11:57:18.502624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d977cd5-7660-4ca4-a0f4-219868543be1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:18.518159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d977cd5-7660-4ca4-a0f4-219868543be1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:18.524116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:18.524757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:18.525189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:18.525470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:18.533923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:18.535565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c55c3e-3b64-4930-be18-975082e5dfa3 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:18.545174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 712/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:20.684315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba155c7-5b06-4567-b10b-f8538d71b2e8 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:20.684846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba155c7-5b06-4567-b10b-f8538d71b2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:20.685415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba155c7-5b06-4567-b10b-f8538d71b2e8 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:20.687404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 657/1369] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:20.692994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b65b-fd00-4cb2-9319-e57ed8ecc4ae None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:20.693848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b65b-fd00-4cb2-9319-e57ed8ecc4ae None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:20.694131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4b65b-fd00-4cb2-9319-e57ed8ecc4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:20.720301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9de4b65b-fd00-4cb2-9319-e57ed8ecc4ae None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:20.721522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9de4b65b-fd00-4cb2-9319-e57ed8ecc4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:20.731622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 713/1370] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:20.786423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba155c7-5b06-4567-b10b-f8538d71b2e8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:20.802169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba155c7-5b06-4567-b10b-f8538d71b2e8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:20.807878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:20.808530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:20.808815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:20.809130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:20.818184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:20.819995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed68e77-d70b-41bb-85eb-e205fee9f683 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:20.830543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 658/1371] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:21.392886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ef1-eb32-46c8-92b3-b7dfdb6bfc09 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:21.393153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ef1-eb32-46c8-92b3-b7dfdb6bfc09 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:21.393356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ef1-eb32-46c8-92b3-b7dfdb6bfc09 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:21.394991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 714/1372] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:21.400466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1dfa6-4a34-4735-977a-d248ee09bffe None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:21.400797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1dfa6-4a34-4735-977a-d248ee09bffe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:21.401319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1dfa6-4a34-4735-977a-d248ee09bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:21.420608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e3f1dfa6-4a34-4735-977a-d248ee09bffe None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:21.421881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e3f1dfa6-4a34-4735-977a-d248ee09bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:21.431075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 659/1373] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:21.795953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6a137-f933-4c06-b70a-38620564fac7 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:21.795953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6a137-f933-4c06-b70a-38620564fac7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:21.795953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6a137-f933-4c06-b70a-38620564fac7 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:21.798264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 715/1374] 10.222.0.55 () {54 vars in 963 bytes} [Tue Jun 28 11:57:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:21.804173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973cc8-3444-4519-a95c-04849d8a4671 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:21.805352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973cc8-3444-4519-a95c-04849d8a4671 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:21.805352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82973cc8-3444-4519-a95c-04849d8a4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:21.855324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-82973cc8-3444-4519-a95c-04849d8a4671 None None] MFA Rules not processed for user `e6c561ff3ab84f99b81eaccc9d9f2cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:21.856627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-82973cc8-3444-4519-a95c-04849d8a4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:21.945767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 660/1375] 10.222.0.55 () {58 vars in 1060 bytes} [Tue Jun 28 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:21.953681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6a137-f933-4c06-b70a-38620564fac7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:21.964758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6a137-f933-4c06-b70a-38620564fac7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:21.968427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:21.969113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:21.969389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:21.969668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:21.980223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:21.981745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d79c0-631b-489d-b6c7-705c4ccfd345 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:21.993526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 716/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:22.912289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba9c1-c194-4adb-93d3-1657060bf608 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:22.912832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba9c1-c194-4adb-93d3-1657060bf608 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:22.912832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba9c1-c194-4adb-93d3-1657060bf608 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:22.914261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 661/1377] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:22.919519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b94ce-95eb-40d5-9b10-3673f78bfe40 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:22.920015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b94ce-95eb-40d5-9b10-3673f78bfe40 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:22.920126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b94ce-95eb-40d5-9b10-3673f78bfe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:22.938012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4b1b94ce-95eb-40d5-9b10-3673f78bfe40 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:22.939366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4b1b94ce-95eb-40d5-9b10-3673f78bfe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:22.947216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 717/1378] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 11:57:22.991784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba9c1-c194-4adb-93d3-1657060bf608 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:23.007027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba9c1-c194-4adb-93d3-1657060bf608 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:23.013043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:23.013771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:23.014117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:23.014393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:23.022712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:23.024317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb106a-fc62-4dfe-94b1-a08b1efa098b service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:23.028124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:23.035009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 662/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:25.109236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab986-4434-4113-9d02-a169dffc14d5 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:25.109825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab986-4434-4113-9d02-a169dffc14d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:25.109825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ab986-4434-4113-9d02-a169dffc14d5 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:25.111126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 718/1380] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 11:57:25.116574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f6fc-5d0d-4be1-8566-9e3d89dd720e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:25.117060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f6fc-5d0d-4be1-8566-9e3d89dd720e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:25.117154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f6fc-5d0d-4be1-8566-9e3d89dd720e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:25.136367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-35d7f6fc-5d0d-4be1-8566-9e3d89dd720e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:25.137593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-35d7f6fc-5d0d-4be1-8566-9e3d89dd720e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:25.146067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 663/1381] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:25.190575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ab986-4434-4113-9d02-a169dffc14d5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:25.206318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ab986-4434-4113-9d02-a169dffc14d5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:25.212026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:25.212687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:25.212938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:25.213249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:25.221488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:25.223085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e9740-d811-43af-9cb4-d2704bd4e549 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:25.232857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 719/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:27.353104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bd30-fe7d-4b64-aa0d-10fc9e8b8ee5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:27.354145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bd30-fe7d-4b64-aa0d-10fc9e8b8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:27.354422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0bd30-fe7d-4b64-aa0d-10fc9e8b8ee5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:27.355815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 664/1383] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:27.361904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6c81d-32d4-4ecd-ad67-7d00b4d2c5bc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:27.362405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6c81d-32d4-4ecd-ad67-7d00b4d2c5bc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:27.362405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6c81d-32d4-4ecd-ad67-7d00b4d2c5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:27.380296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6dc6c81d-32d4-4ecd-ad67-7d00b4d2c5bc None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:27.381522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6dc6c81d-32d4-4ecd-ad67-7d00b4d2c5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:27.389725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 720/1384] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:27.431261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bd30-fe7d-4b64-aa0d-10fc9e8b8ee5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:27.447049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0bd30-fe7d-4b64-aa0d-10fc9e8b8ee5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:27.483811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:27.483811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:27.483811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:27.483811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:27.484284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:27.484284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d6e7b-89e6-4556-9a1f-091bc4e61417 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:27.491458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 665/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:29.550714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e73bb-a5b5-4a10-981e-24fbdc30f831 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:29.551327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e73bb-a5b5-4a10-981e-24fbdc30f831 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:29.551327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e73bb-a5b5-4a10-981e-24fbdc30f831 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:29.552661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 721/1386] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:29.557970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee516-6611-4254-b337-2999728c8535 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:29.558411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee516-6611-4254-b337-2999728c8535 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:29.558533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ee516-6611-4254-b337-2999728c8535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:29.577394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-947ee516-6611-4254-b337-2999728c8535 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:29.578594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-947ee516-6611-4254-b337-2999728c8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:29.586578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 666/1387] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:29.632740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e73bb-a5b5-4a10-981e-24fbdc30f831 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:29.650337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e73bb-a5b5-4a10-981e-24fbdc30f831 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:29.657910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:29.659065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:29.659750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:29.660382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:29.673225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:29.675151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00874f4f-cfab-447e-bfd1-d5bfb8f4b93c service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:29.687409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 722/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:32.037626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfe85-3476-4210-a140-7d28d6c88faa None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:32.038380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfe85-3476-4210-a140-7d28d6c88faa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:32.038380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dfe85-3476-4210-a140-7d28d6c88faa None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:32.039498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 667/1389] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:32.045197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d823-0e42-4c3a-b4e6-1861affbd566 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:32.045723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d823-0e42-4c3a-b4e6-1861affbd566 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:32.045921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d823-0e42-4c3a-b4e6-1861affbd566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:32.069610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4262d823-0e42-4c3a-b4e6-1861affbd566 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:32.071283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4262d823-0e42-4c3a-b4e6-1861affbd566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:32.082610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 723/1390] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:32.135691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfe85-3476-4210-a140-7d28d6c88faa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:32.162757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dfe85-3476-4210-a140-7d28d6c88faa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:32.170515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:32.171434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:32.171833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:32.172253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:32.184556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:32.187035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d1989-529a-42d9-a609-a93edd92c8ec service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:32.201698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 668/1391] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:34.312315 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f86f77-aba7-4ff4-830f-d803fb3a9817 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:34.312975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f86f77-aba7-4ff4-830f-d803fb3a9817 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:34.312975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f86f77-aba7-4ff4-830f-d803fb3a9817 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:34.317128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 724/1392] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:34.326627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d71cc3-d9ee-4c64-a608-7d2bc72c06dc None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:34.327400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d71cc3-d9ee-4c64-a608-7d2bc72c06dc None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:34.327734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d71cc3-d9ee-4c64-a608-7d2bc72c06dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:34.349853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-49d71cc3-d9ee-4c64-a608-7d2bc72c06dc None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:34.351110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-49d71cc3-d9ee-4c64-a608-7d2bc72c06dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:34.360587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 669/1393] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:34.408419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f86f77-aba7-4ff4-830f-d803fb3a9817 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:34.428410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f86f77-aba7-4ff4-830f-d803fb3a9817 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:34.435851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:34.436567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:34.436856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:34.437165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:34.448370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:34.450367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c18a21-c81f-4bdd-9ac4-18c6cdc2c42d service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:34.461250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 725/1394] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:36.683533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a25e-aed3-42a5-acba-9468ce1d2826 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:36.684372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a25e-aed3-42a5-acba-9468ce1d2826 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:36.684650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7a25e-aed3-42a5-acba-9468ce1d2826 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:36.686437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 670/1395] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:36.692116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643086-fc38-4f68-a5c6-1a44844f58aa None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:36.692642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643086-fc38-4f68-a5c6-1a44844f58aa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:36.692642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643086-fc38-4f68-a5c6-1a44844f58aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:36.712055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70643086-fc38-4f68-a5c6-1a44844f58aa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:36.713340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70643086-fc38-4f68-a5c6-1a44844f58aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:36.722334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 726/1396] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:36.768318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7a25e-aed3-42a5-acba-9468ce1d2826 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:36.786804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7a25e-aed3-42a5-acba-9468ce1d2826 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:36.792841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:36.793544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:36.793825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:36.794092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:36.803444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:36.805099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39c7d2-d8b1-4e18-a493-6cd84008ac08 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:36.814719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 671/1397] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:37.108114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135ebe1-24f0-4a04-8e59-a262b95aa20b None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:37.108384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135ebe1-24f0-4a04-8e59-a262b95aa20b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:37.108661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135ebe1-24f0-4a04-8e59-a262b95aa20b None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:37.110141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 727/1398] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:37.115580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11bc10-244d-4aa1-98a6-3c52b90edb72 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:37.115946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11bc10-244d-4aa1-98a6-3c52b90edb72 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:37.116228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11bc10-244d-4aa1-98a6-3c52b90edb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:37.135272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7b11bc10-244d-4aa1-98a6-3c52b90edb72 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:37.136513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7b11bc10-244d-4aa1-98a6-3c52b90edb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:37.146101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 672/1399] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:37.203249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78700-703f-4705-8361-6522e7feadf4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:37.203538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78700-703f-4705-8361-6522e7feadf4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:37.203812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad78700-703f-4705-8361-6522e7feadf4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:37.205310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 728/1400] 10.222.0.55 () {54 vars in 962 bytes} [Tue Jun 28 11:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:37.210583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d999-60a7-475d-b7f4-d30efa1dabb6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:37.210870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d999-60a7-475d-b7f4-d30efa1dabb6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:37.211150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d999-60a7-475d-b7f4-d30efa1dabb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:37.261265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e3f8d999-60a7-475d-b7f4-d30efa1dabb6 None None] MFA Rules not processed for user `0f5e14d9d362461dae34b8330718002b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:37.262477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e3f8d999-60a7-475d-b7f4-d30efa1dabb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:37.324840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 673/1401] 10.222.0.55 () {58 vars in 1059 bytes} [Tue Jun 28 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:37.332630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad78700-703f-4705-8361-6522e7feadf4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:37.344956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad78700-703f-4705-8361-6522e7feadf4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:37.348944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:37.349606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:37.349853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:37.350160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:37.359529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:37.360938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef3258-2430-416d-b42d-c59981ddf444 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:37.370443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 729/1402] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:38.097171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4cf5-ba50-4512-bc7e-064b727ec6f9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.097697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4cf5-ba50-4512-bc7e-064b727ec6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.097697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4cf5-ba50-4512-bc7e-064b727ec6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.149284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-889a4cf5-ba50-4512-bc7e-064b727ec6f9 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:38.150494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-889a4cf5-ba50-4512-bc7e-064b727ec6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:38.190369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:38.214846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 674/1403] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:38.259748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e573-73fe-42af-8131-222cd4a55688 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.260250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e573-73fe-42af-8131-222cd4a55688 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.260355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e573-73fe-42af-8131-222cd4a55688 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.261906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 730/1404] 10.222.0.55 () {54 vars in 942 bytes} [Tue Jun 28 11:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:38.268375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33120af-87e5-441e-b097-3e2926e993cd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.268681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33120af-87e5-441e-b097-3e2926e993cd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.268957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33120af-87e5-441e-b097-3e2926e993cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.318184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c33120af-87e5-441e-b097-3e2926e993cd None None] MFA Rules not processed for user `ee9dc1ba0084428c9da313ebda0458e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:38.319293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c33120af-87e5-441e-b097-3e2926e993cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:38.396973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 675/1405] 10.222.0.55 () {58 vars in 1039 bytes} [Tue Jun 28 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:38.406483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734e573-73fe-42af-8131-222cd4a55688 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:38.421130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734e573-73fe-42af-8131-222cd4a55688 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:38.426634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:38.427295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.427574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.427849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.436618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:38.438263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac302-691e-4101-9796-d7a1037f4cf6 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:38.448301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 731/1406] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 11:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 11:57:38.902664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48499-3a2a-4020-9506-f5234ca2d328 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.902930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48499-3a2a-4020-9506-f5234ca2d328 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.903210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48499-3a2a-4020-9506-f5234ca2d328 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.904705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 676/1407] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:57:38.910220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9506-1d06-4114-ac43-86f9497ed781 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:38.910627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9506-1d06-4114-ac43-86f9497ed781 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:38.911008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe9506-1d06-4114-ac43-86f9497ed781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:38.917805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=67 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:38.931652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-96fe9506-1d06-4114-ac43-86f9497ed781 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:57:38.933122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-96fe9506-1d06-4114-ac43-86f9497ed781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:57:38.941391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 732/1408] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:57:38.987307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf48499-3a2a-4020-9506-f5234ca2d328 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:57:39.002569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf48499-3a2a-4020-9506-f5234ca2d328 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:57:39.008053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:57:39.008700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:57:39.009013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:57:39.009286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:57:39.018165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:57:39.019774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa196-4ea3-4646-9b71-dad3eebdc458 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:57:39.029635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 677/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:57:48.926007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:57:49.025071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=59 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:58:01.152601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c73d5-6848-4400-8a91-76d5eed7e5cf None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:01.153291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c73d5-6848-4400-8a91-76d5eed7e5cf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:01.153291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c73d5-6848-4400-8a91-76d5eed7e5cf None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:01.154591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 733/1410] 10.222.0.55 () {54 vars in 918 bytes} [Tue Jun 28 11:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:01.162393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227255e0-910a-48e4-9146-75c01d9da4ef None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:01.162877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227255e0-910a-48e4-9146-75c01d9da4ef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:01.162985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227255e0-910a-48e4-9146-75c01d9da4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:01.184514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-227255e0-910a-48e4-9146-75c01d9da4ef None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:01.185889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-227255e0-910a-48e4-9146-75c01d9da4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:01.196212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 678/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Jun 28 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:01.213751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb1de4-7bd0-42d5-9910-b7a9961e8ac3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:01.214036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb1de4-7bd0-42d5-9910-b7a9961e8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:01.214305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb1de4-7bd0-42d5-9910-b7a9961e8ac3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:01.215689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 734/1412] 10.222.0.55 () {54 vars in 918 bytes} [Tue Jun 28 11:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:01.221783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2424-b7dd-4b44-84f7-91890c65ad2a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:01.222154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2424-b7dd-4b44-84f7-91890c65ad2a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:01.222478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c2424-b7dd-4b44-84f7-91890c65ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:01.245744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c92c2424-b7dd-4b44-84f7-91890c65ad2a None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:01.246944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c92c2424-b7dd-4b44-84f7-91890c65ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:01.258031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 679/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Jun 28 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:03.775822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372651-526b-4e5c-809e-0d90bff63b2c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:03.776649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372651-526b-4e5c-809e-0d90bff63b2c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:03.776649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49372651-526b-4e5c-809e-0d90bff63b2c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:03.778954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 735/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:03.784940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecddd8-6727-42c5-9a2b-9e82b8121cc3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:03.785429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecddd8-6727-42c5-9a2b-9e82b8121cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:03.785540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecddd8-6727-42c5-9a2b-9e82b8121cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:03.805725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-03ecddd8-6727-42c5-9a2b-9e82b8121cc3 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:03.806932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-03ecddd8-6727-42c5-9a2b-9e82b8121cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:03.816805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 680/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:03.863272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372651-526b-4e5c-809e-0d90bff63b2c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:03.883557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49372651-526b-4e5c-809e-0d90bff63b2c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:03.889380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:03.890023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:03.890298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:03.890572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:03.898862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:03.900457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b653-b483-4a84-878b-9aa17658e767 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:03.909973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 736/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:04.813486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6363-a248-4d7d-9599-5b4ab84a134e None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:04.814095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6363-a248-4d7d-9599-5b4ab84a134e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:04.814095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6363-a248-4d7d-9599-5b4ab84a134e None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:04.815483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 681/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:04.821439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b32c5a-f892-4c27-81a5-d1ce9b5803bf None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:04.821918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b32c5a-f892-4c27-81a5-d1ce9b5803bf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:04.822021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b32c5a-f892-4c27-81a5-d1ce9b5803bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:04.847748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-15b32c5a-f892-4c27-81a5-d1ce9b5803bf None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:04.849376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-15b32c5a-f892-4c27-81a5-d1ce9b5803bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:04.858247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 737/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:05.007890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b6363-a248-4d7d-9599-5b4ab84a134e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:05.024816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b6363-a248-4d7d-9599-5b4ab84a134e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:05.034783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:05.035441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:05.035720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:05.035990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:05.045159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:05.046855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734a9aa-a846-472f-ade0-08af05f68307 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:05.057543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 682/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:05.950961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293223-a362-4fda-8d69-42a0e20bfd8f None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:05.951987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293223-a362-4fda-8d69-42a0e20bfd8f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:05.951987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293223-a362-4fda-8d69-42a0e20bfd8f None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:05.953421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 738/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:05.958736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fa4d-21fc-44f7-81df-56cfdf3d989a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:05.959193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fa4d-21fc-44f7-81df-56cfdf3d989a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:05.959299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fa4d-21fc-44f7-81df-56cfdf3d989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:05.984244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b711fa4d-21fc-44f7-81df-56cfdf3d989a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:05.985490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b711fa4d-21fc-44f7-81df-56cfdf3d989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:05.994913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 683/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:06.157381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe293223-a362-4fda-8d69-42a0e20bfd8f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:06.174979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe293223-a362-4fda-8d69-42a0e20bfd8f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:06.181426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:06.182064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:06.182344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:06.182612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:06.191332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:06.192937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e159d-aa6f-468b-8edb-c65bb5638124 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:06.202663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 739/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:07.090678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbbb6-76b4-48dc-8095-9acc0e5b8aa3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:07.091286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbbb6-76b4-48dc-8095-9acc0e5b8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:07.091286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbbb6-76b4-48dc-8095-9acc0e5b8aa3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:07.093080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 684/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:07.099625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1065598-58c4-46b5-a39d-56febdcc2aba None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:07.100084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1065598-58c4-46b5-a39d-56febdcc2aba None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:07.100187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1065598-58c4-46b5-a39d-56febdcc2aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:07.127798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c1065598-58c4-46b5-a39d-56febdcc2aba None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:07.129046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c1065598-58c4-46b5-a39d-56febdcc2aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:07.138122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 740/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:07.300772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dbbb6-76b4-48dc-8095-9acc0e5b8aa3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:07.319407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dbbb6-76b4-48dc-8095-9acc0e5b8aa3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:07.327852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:07.328510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:07.328786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:07.329097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:07.338902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:07.341381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc2b5f-9e67-4f4b-a2f1-95a3115e4b91 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:07.353057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 685/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:08.238908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e07cfc-b786-4fa0-9791-1160f9c5945c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:08.239453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e07cfc-b786-4fa0-9791-1160f9c5945c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:08.239453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e07cfc-b786-4fa0-9791-1160f9c5945c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:08.240756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 741/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:08.249449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe903-ee06-4c0b-a9a7-7e1713d473aa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:08.250390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe903-ee06-4c0b-a9a7-7e1713d473aa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:08.250390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fe903-ee06-4c0b-a9a7-7e1713d473aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:08.275156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-471fe903-ee06-4c0b-a9a7-7e1713d473aa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:08.276359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-471fe903-ee06-4c0b-a9a7-7e1713d473aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:08.285801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 686/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:08.438131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e07cfc-b786-4fa0-9791-1160f9c5945c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:08.461483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e07cfc-b786-4fa0-9791-1160f9c5945c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:08.470556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:08.471197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:08.471480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:08.471747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:08.480600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:08.482245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45a5e5-ba78-474a-9cc7-a8188a1c7d8f service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:08.491848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 742/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:09.316206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc595-6799-4d4d-b8c2-b053d82da949 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:09.316819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc595-6799-4d4d-b8c2-b053d82da949 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:09.316819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc595-6799-4d4d-b8c2-b053d82da949 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:09.318149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 687/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:09.324160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946e05-ab46-42ab-85c9-33affd847f17 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:09.324968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946e05-ab46-42ab-85c9-33affd847f17 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:09.325184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd946e05-ab46-42ab-85c9-33affd847f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:09.350612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bd946e05-ab46-42ab-85c9-33affd847f17 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:09.351846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bd946e05-ab46-42ab-85c9-33affd847f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:09.360817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 743/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:09.512759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc595-6799-4d4d-b8c2-b053d82da949 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:09.528124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc595-6799-4d4d-b8c2-b053d82da949 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:09.533677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:09.534315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:09.534590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:09.534861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:09.543195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:09.544784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450b65d-02d3-4104-82b3-c9e0bb198872 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:09.554277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 688/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:10.375097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3aa4-5b4d-401d-a5eb-762dfede4452 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:10.375998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3aa4-5b4d-401d-a5eb-762dfede4452 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:10.375998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3aa4-5b4d-401d-a5eb-762dfede4452 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:10.378211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 744/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:10.384379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139de8e-9016-47f0-807e-9719def7bc54 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:10.385144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139de8e-9016-47f0-807e-9719def7bc54 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:10.385326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139de8e-9016-47f0-807e-9719def7bc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:10.405897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1139de8e-9016-47f0-807e-9719def7bc54 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:10.407121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1139de8e-9016-47f0-807e-9719def7bc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:10.417339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 689/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:10.571148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af3aa4-5b4d-401d-a5eb-762dfede4452 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:10.596304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af3aa4-5b4d-401d-a5eb-762dfede4452 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:10.603597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:10.604235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:10.604555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:10.604819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:10.613957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:10.615542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c20f6-2784-47cf-a6f6-e9eacaa50af1 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:10.626038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 745/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:11.433630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d84a-2f8c-4725-a13d-f356224669e9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:11.434248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d84a-2f8c-4725-a13d-f356224669e9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:11.434248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d84a-2f8c-4725-a13d-f356224669e9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:11.435572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 690/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:11.442281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320918-f3d6-4030-a3b6-136be51016f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:11.442753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320918-f3d6-4030-a3b6-136be51016f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:11.442873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320918-f3d6-4030-a3b6-136be51016f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:11.470584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-82320918-f3d6-4030-a3b6-136be51016f2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:11.471783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-82320918-f3d6-4030-a3b6-136be51016f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:11.481376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 746/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:11.636188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d84a-2f8c-4725-a13d-f356224669e9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:11.653890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d84a-2f8c-4725-a13d-f356224669e9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:11.659846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:11.660467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:11.660745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:11.661053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:11.668999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:11.670769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e67504-7db8-4ca1-9afe-261e68d5c81e service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:11.680498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 691/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:12.498754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e3a1-0bb0-4e6e-b5bd-bd010d1da7dd None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:12.499634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e3a1-0bb0-4e6e-b5bd-bd010d1da7dd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:12.499811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6e3a1-0bb0-4e6e-b5bd-bd010d1da7dd None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:12.501295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 747/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 11:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:12.507289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c95-a28b-4851-be0a-94607a72165a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:12.508005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c95-a28b-4851-be0a-94607a72165a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:12.508218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c95-a28b-4851-be0a-94607a72165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:12.529709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-45450c95-a28b-4851-be0a-94607a72165a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:12.531413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-45450c95-a28b-4851-be0a-94607a72165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:12.539998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 692/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:12.693917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6e3a1-0bb0-4e6e-b5bd-bd010d1da7dd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:12.712035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6e3a1-0bb0-4e6e-b5bd-bd010d1da7dd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:12.718959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:12.719611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:12.719896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:12.720169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:12.730058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:12.731705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfb181-5ccc-485f-a065-6766d1a41bdc service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:12.741598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 748/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:14.648480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c3b56-f287-4612-885d-8edca0c3791c None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:14.649060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c3b56-f287-4612-885d-8edca0c3791c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:14.649189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c3b56-f287-4612-885d-8edca0c3791c None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:14.650501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 693/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:14.656643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd735e-772f-4ce0-8e90-61397602486c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:14.657371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd735e-772f-4ce0-8e90-61397602486c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:14.657518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd735e-772f-4ce0-8e90-61397602486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:14.680262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-10fd735e-772f-4ce0-8e90-61397602486c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:14.681499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-10fd735e-772f-4ce0-8e90-61397602486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:14.690123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 749/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:14.728566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c3b56-f287-4612-885d-8edca0c3791c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:14.741087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c3b56-f287-4612-885d-8edca0c3791c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:14.745401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:14.746036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:14.746315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:14.746586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:14.756771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:14.758244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298a17-78a3-43df-8f2a-929b94d0c2fa service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:14.768071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 694/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:16.943336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b2c2-6925-42d8-92b5-803aa6a7bd64 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:16.944229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b2c2-6925-42d8-92b5-803aa6a7bd64 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:16.944229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b2c2-6925-42d8-92b5-803aa6a7bd64 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:16.946513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 750/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:16.954522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b1bd9-6382-4d1e-b595-924d095b3e32 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:16.954945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b1bd9-6382-4d1e-b595-924d095b3e32 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:16.955048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b1bd9-6382-4d1e-b595-924d095b3e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:16.974897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9c6b1bd9-6382-4d1e-b595-924d095b3e32 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:16.976117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9c6b1bd9-6382-4d1e-b595-924d095b3e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:16.987910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 695/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:17.037824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b2c2-6925-42d8-92b5-803aa6a7bd64 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:17.050730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b2c2-6925-42d8-92b5-803aa6a7bd64 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:17.054911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:17.055577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:17.055837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:17.056103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:17.065853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:17.067283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a81ac-8767-42dd-add4-630c04ea4e50 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:17.077484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 751/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:19.425394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd4775-5e3a-42f1-8983-727b125dcedd None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:19.425954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd4775-5e3a-42f1-8983-727b125dcedd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:19.425954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd4775-5e3a-42f1-8983-727b125dcedd None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:19.427384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 696/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:19.432893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d3bcb-5b12-4845-8a39-d54a686ce867 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:19.433359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d3bcb-5b12-4845-8a39-d54a686ce867 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:19.433494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d3bcb-5b12-4845-8a39-d54a686ce867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:19.454716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-693d3bcb-5b12-4845-8a39-d54a686ce867 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:19.455918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-693d3bcb-5b12-4845-8a39-d54a686ce867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:19.463731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 752/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:19.503957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd4775-5e3a-42f1-8983-727b125dcedd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:19.516755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd4775-5e3a-42f1-8983-727b125dcedd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:19.520348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:19.521031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:19.521315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:19.521589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:19.531023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:19.532508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091664d-1d0c-47fb-a1c9-eecfc82349dd service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:19.542850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 697/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:21.829754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a5ca-57b1-4410-9d4f-0f655aca6e51 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:21.830371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a5ca-57b1-4410-9d4f-0f655aca6e51 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:21.830470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a5ca-57b1-4410-9d4f-0f655aca6e51 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:21.831885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 753/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:21.837546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741732c-3517-4fe2-8a3d-ec9eb0c40993 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:21.838014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741732c-3517-4fe2-8a3d-ec9eb0c40993 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:21.838117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741732c-3517-4fe2-8a3d-ec9eb0c40993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:21.857780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3741732c-3517-4fe2-8a3d-ec9eb0c40993 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:21.859707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3741732c-3517-4fe2-8a3d-ec9eb0c40993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:21.868869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 698/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:21.927666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a5ca-57b1-4410-9d4f-0f655aca6e51 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:21.942062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a5ca-57b1-4410-9d4f-0f655aca6e51 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:21.945924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:21.946537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:21.946804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:21.947105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:21.957020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:21.958454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe99f50-0266-4b80-9f93-c14a8ce00d3a service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:21.967891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 754/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:24.171933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb0ecf-c062-4ea3-a9de-ae46f8a226e9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:24.172794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb0ecf-c062-4ea3-a9de-ae46f8a226e9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:24.172794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb0ecf-c062-4ea3-a9de-ae46f8a226e9 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:24.175071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 699/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:24.180608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a196e8-7c4a-474f-a07a-32eb06bf2fc0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:24.181137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a196e8-7c4a-474f-a07a-32eb06bf2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:24.181282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a196e8-7c4a-474f-a07a-32eb06bf2fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:24.199934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-51a196e8-7c4a-474f-a07a-32eb06bf2fc0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:24.201169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51a196e8-7c4a-474f-a07a-32eb06bf2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:24.209707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 755/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:24.248366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb0ecf-c062-4ea3-a9de-ae46f8a226e9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:24.267163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb0ecf-c062-4ea3-a9de-ae46f8a226e9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:24.271538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:24.272181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:24.272460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:24.272731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:24.282975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:24.284425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a5e67-a5d9-4430-8884-e048dcc4a2cc service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:24.294525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 700/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:26.420509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd49cf1-b09d-434f-8c5a-a58379f1b371 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:26.421668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd49cf1-b09d-434f-8c5a-a58379f1b371 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:26.421668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd49cf1-b09d-434f-8c5a-a58379f1b371 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:26.423561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 756/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:26.430092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce471e-f174-419a-b3f5-936759dc33db None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:26.430535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce471e-f174-419a-b3f5-936759dc33db None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:26.430646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce471e-f174-419a-b3f5-936759dc33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:26.452203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-83ce471e-f174-419a-b3f5-936759dc33db None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:26.453389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-83ce471e-f174-419a-b3f5-936759dc33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:26.463534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 701/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:26.502326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd49cf1-b09d-434f-8c5a-a58379f1b371 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:26.514319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd49cf1-b09d-434f-8c5a-a58379f1b371 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:26.518455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:26.519106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:26.519389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:26.519701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:26.531090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:26.532537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fd3be-6dca-4e61-8288-1e34831a1106 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:26.542736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 757/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:28.644160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530824b3-f20a-4a4f-aa3a-2d9ddf6d1dc3 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:28.644945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530824b3-f20a-4a4f-aa3a-2d9ddf6d1dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:28.645107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530824b3-f20a-4a4f-aa3a-2d9ddf6d1dc3 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:28.647350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 702/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:28.652894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b9ae-13c9-4e32-834b-ca6f203112a7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:28.653363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b9ae-13c9-4e32-834b-ca6f203112a7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:28.653471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408b9ae-13c9-4e32-834b-ca6f203112a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:28.672429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0408b9ae-13c9-4e32-834b-ca6f203112a7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:28.673672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0408b9ae-13c9-4e32-834b-ca6f203112a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:28.682306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 758/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:28.720709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530824b3-f20a-4a4f-aa3a-2d9ddf6d1dc3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:28.738970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530824b3-f20a-4a4f-aa3a-2d9ddf6d1dc3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:28.744568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:28.745539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:28.745908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:28.746272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:28.756689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:28.758167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fa377-6b2a-4963-945b-63cd9cff682b service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:28.768010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 703/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:30.844098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf7b5-e171-410e-8a25-6098bf8ba90c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:30.845068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf7b5-e171-410e-8a25-6098bf8ba90c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:30.845068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bf7b5-e171-410e-8a25-6098bf8ba90c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:30.847268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 759/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:30.853174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847643-089c-4066-9f0e-ef31100c57ff None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:30.853604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847643-089c-4066-9f0e-ef31100c57ff None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:30.853723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847643-089c-4066-9f0e-ef31100c57ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:30.860647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=79 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:58:30.875069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-55847643-089c-4066-9f0e-ef31100c57ff None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:30.876269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-55847643-089c-4066-9f0e-ef31100c57ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:30.884753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 704/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:30.922357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bf7b5-e171-410e-8a25-6098bf8ba90c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:30.929225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=73 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:58:30.936520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bf7b5-e171-410e-8a25-6098bf8ba90c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:30.940238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:30.940878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:30.941199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:30.941471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:30.950403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:30.951960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5b22-e35f-45ca-bb39-27539b533226 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:30.961413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 760/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:33.043855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2a35c-874a-4fd7-97d5-bb3dab4cd63d None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:33.044760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2a35c-874a-4fd7-97d5-bb3dab4cd63d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:33.044760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2a35c-874a-4fd7-97d5-bb3dab4cd63d None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:33.047078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 705/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:33.053659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85448c8-21d2-4e47-9bec-f65de3a14a09 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:33.054354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85448c8-21d2-4e47-9bec-f65de3a14a09 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:33.055509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85448c8-21d2-4e47-9bec-f65de3a14a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:33.077825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b85448c8-21d2-4e47-9bec-f65de3a14a09 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:33.079097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b85448c8-21d2-4e47-9bec-f65de3a14a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:33.089012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 761/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:33.128805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2a35c-874a-4fd7-97d5-bb3dab4cd63d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:33.140570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2a35c-874a-4fd7-97d5-bb3dab4cd63d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:33.146290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:33.147254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:33.147649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:33.148040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:33.158882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:33.160333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c46bd6-f96b-44d2-ae81-ac8d69c50b4c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:33.171377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 706/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:35.286786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c60ca-9f21-4a00-b3f7-c8b37b2dc2a0 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:35.287690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c60ca-9f21-4a00-b3f7-c8b37b2dc2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:35.287690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c60ca-9f21-4a00-b3f7-c8b37b2dc2a0 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:35.290034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 762/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 11:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:35.295653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfd27-472a-40fb-ba9d-212d7bd4c117 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:35.296108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfd27-472a-40fb-ba9d-212d7bd4c117 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:35.296212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcfd27-472a-40fb-ba9d-212d7bd4c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:35.316625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-98fcfd27-472a-40fb-ba9d-212d7bd4c117 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:35.317773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-98fcfd27-472a-40fb-ba9d-212d7bd4c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:35.326174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 707/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 11:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:35.367070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c60ca-9f21-4a00-b3f7-c8b37b2dc2a0 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:35.381502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c60ca-9f21-4a00-b3f7-c8b37b2dc2a0 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:35.385931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:35.386579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:35.386983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:35.387264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:35.397185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:35.398635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682f28f-40b9-4fab-8e7a-9518edb21887 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:35.408370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 763/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:45.310335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=11 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:58:45.403912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=14 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:58:55.509554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b27f-1198-46a7-8601-2e1f1e10a990 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:55.510193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b27f-1198-46a7-8601-2e1f1e10a990 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:55.510193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b27f-1198-46a7-8601-2e1f1e10a990 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:55.511370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 708/1471] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:55.518740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5476-e3b1-420c-8554-1fbd0f62a891 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:55.519199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5476-e3b1-420c-8554-1fbd0f62a891 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:55.519305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f5476-e3b1-420c-8554-1fbd0f62a891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:55.541748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-836f5476-e3b1-420c-8554-1fbd0f62a891 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:55.543003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-836f5476-e3b1-420c-8554-1fbd0f62a891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:55.553569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 764/1472] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:55.608364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926519-288e-49da-950d-c52fc501855a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:55.608650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926519-288e-49da-950d-c52fc501855a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:55.608949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba926519-288e-49da-950d-c52fc501855a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:55.610381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 709/1473] 10.222.0.55 () {54 vars in 962 bytes} [Tue Jun 28 11:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:55.615370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217724b-5d48-4876-b900-8831eefd70cc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:55.615653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217724b-5d48-4876-b900-8831eefd70cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:55.615923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217724b-5d48-4876-b900-8831eefd70cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:55.634955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1217724b-5d48-4876-b900-8831eefd70cc None None] MFA Rules not processed for user `0f5e14d9d362461dae34b8330718002b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:55.636118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1217724b-5d48-4876-b900-8831eefd70cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:55.643612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 765/1474] 10.222.0.55 () {58 vars in 1059 bytes} [Tue Jun 28 11:58:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:55.650860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba926519-288e-49da-950d-c52fc501855a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:55.664155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba926519-288e-49da-950d-c52fc501855a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:55.668060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:55.668698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:55.669021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:55.669296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:55.679974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:55.681480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785a685-1f31-4b1d-acea-0a677cb87c92 service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:55.691900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 710/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:58:57.824385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12330f-2ea1-4004-ac51-462c26348dd4 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:57.825011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12330f-2ea1-4004-ac51-462c26348dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:57.825011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12330f-2ea1-4004-ac51-462c26348dd4 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:57.826350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 766/1476] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:57.831790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451b7b-6425-4af6-bf82-650f53100089 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:57.832242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451b7b-6425-4af6-bf82-650f53100089 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:57.832371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451b7b-6425-4af6-bf82-650f53100089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:57.851415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ab451b7b-6425-4af6-bf82-650f53100089 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:58:57.852605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ab451b7b-6425-4af6-bf82-650f53100089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:58:57.863327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 711/1477] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:58:57.891179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12330f-2ea1-4004-ac51-462c26348dd4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:58:57.905443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12330f-2ea1-4004-ac51-462c26348dd4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:58:57.910206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:58:57.910872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:57.911157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:57.911432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:57.912216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:58:57.914041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 11:58:57.914041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 11:58:57.914041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 11:58:57.915201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec48df-0dd2-413c-ac1e-85e039148d0a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:58:57.935190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 767/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Jun 28 11:58:57 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 11:58:59.982843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d4ec-2d22-448a-b69f-f3d18a411438 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:59.983431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d4ec-2d22-448a-b69f-f3d18a411438 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:59.983431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50d4ec-2d22-448a-b69f-f3d18a411438 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:58:59.984975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 712/1479] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:58:59.991165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8acdd-d266-40d1-a968-c3b9297d7ec1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:58:59.991664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8acdd-d266-40d1-a968-c3b9297d7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:58:59.991664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8acdd-d266-40d1-a968-c3b9297d7ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:00.011437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dee8acdd-d266-40d1-a968-c3b9297d7ec1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:59:00.012669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dee8acdd-d266-40d1-a968-c3b9297d7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:59:00.021994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 768/1480] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:59:00.070295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50d4ec-2d22-448a-b69f-f3d18a411438 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:59:00.085684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50d4ec-2d22-448a-b69f-f3d18a411438 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:59:00.091649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:59:00.092290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:00.092572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:00.092926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:00.101903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:59:00.103523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039587-5554-44c5-9b24-6b9d05dc97de service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:59:00.116571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 713/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:59:02.333886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647ff4-54ac-4953-af21-13e0a1f17f3e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:02.334824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647ff4-54ac-4953-af21-13e0a1f17f3e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:02.334824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647ff4-54ac-4953-af21-13e0a1f17f3e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:02.336941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 769/1482] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:59:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:59:02.345741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb873-f547-4c41-bf2d-040b0c445a0c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:02.346198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb873-f547-4c41-bf2d-040b0c445a0c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:02.346308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb873-f547-4c41-bf2d-040b0c445a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:02.375968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-454fb873-f547-4c41-bf2d-040b0c445a0c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:59:02.377220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-454fb873-f547-4c41-bf2d-040b0c445a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:59:02.386709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 714/1483] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:59:02.426733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1647ff4-54ac-4953-af21-13e0a1f17f3e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:59:02.443831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1647ff4-54ac-4953-af21-13e0a1f17f3e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:59:02.448497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:59:02.449169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:02.449446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:02.449717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:02.460557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:59:02.462062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c541e59-3aff-4f59-96c2-ecc241b79f04 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:59:02.472392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 770/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:59:04.578393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b8717-ec79-493a-b18a-dd262956cc27 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:04.579293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b8717-ec79-493a-b18a-dd262956cc27 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:04.579293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b8717-ec79-493a-b18a-dd262956cc27 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:04.581617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 715/1485] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:59:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:59:04.588035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21d63-15a3-4f43-9497-f9fefdd49299 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:04.588507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21d63-15a3-4f43-9497-f9fefdd49299 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:04.588613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21d63-15a3-4f43-9497-f9fefdd49299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:04.610033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2da21d63-15a3-4f43-9497-f9fefdd49299 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:59:04.611243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2da21d63-15a3-4f43-9497-f9fefdd49299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:59:04.620998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 771/1486] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:59:04.659284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b8717-ec79-493a-b18a-dd262956cc27 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:59:04.673317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b8717-ec79-493a-b18a-dd262956cc27 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:59:04.677876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:59:04.678522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:04.678805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:04.679077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:04.689966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:59:04.691424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbcb17-42c0-4ade-b3f4-5ed94f86a10d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:59:04.701876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 716/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:59:06.907054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a4c7-b983-49e5-9a90-d0387d486ba3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:06.907630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a4c7-b983-49e5-9a90-d0387d486ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:06.907630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4a4c7-b983-49e5-9a90-d0387d486ba3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:06.909044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 772/1488] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:59:06.915567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f5f0-78c8-420a-9ae2-e112ee29807a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:06.916027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f5f0-78c8-420a-9ae2-e112ee29807a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:06.916134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f5f0-78c8-420a-9ae2-e112ee29807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:06.942975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ec58f5f0-78c8-420a-9ae2-e112ee29807a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:59:06.944186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ec58f5f0-78c8-420a-9ae2-e112ee29807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:59:06.953070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 717/1489] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:59:06.997995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4a4c7-b983-49e5-9a90-d0387d486ba3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:59:07.011381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4a4c7-b983-49e5-9a90-d0387d486ba3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:59:07.016651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:59:07.017336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:07.017600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:07.017872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:07.027765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:59:07.029227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3a47-d76d-4e9c-9b34-4f60172a28b0 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:59:07.039976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 773/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:59:09.289024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4e36-e35c-401e-b570-f15a6cde6cbd None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:09.289633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4e36-e35c-401e-b570-f15a6cde6cbd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:09.289633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4e36-e35c-401e-b570-f15a6cde6cbd None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:09.290906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 718/1491] 10.222.0.55 () {54 vars in 926 bytes} [Tue Jun 28 11:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 11:59:09.297494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cec24-a10b-4ca6-945a-1e517582c973 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:09.298290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cec24-a10b-4ca6-945a-1e517582c973 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:09.298455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cec24-a10b-4ca6-945a-1e517582c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:09.323024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b17cec24-a10b-4ca6-945a-1e517582c973 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 11:59:09.324240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b17cec24-a10b-4ca6-945a-1e517582c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 11:59:09.333574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 774/1492] 10.222.0.55 () {58 vars in 1023 bytes} [Tue Jun 28 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 11:59:09.377209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e4e36-e35c-401e-b570-f15a6cde6cbd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 11:59:09.394085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e4e36-e35c-401e-b570-f15a6cde6cbd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 11:59:09.398251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 11:59:09.398895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 11:59:09.399175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 11:59:09.399460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 11:59:09.408873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 11:59:09.410350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a3200-3be0-440a-b233-121f93945390 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 11:59:09.420385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 719/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 11:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 11:59:19.317634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=33 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 11:59:19.416024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=33 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:00:41.490422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.491065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.491065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.514845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:00:41.516043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:41.525776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 775/1494] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:00:41.534284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.534804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.534804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.554126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:00:41.555299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:41.564608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 720/1495] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:00:41.569165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.578881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb2a5-5578-4c02-9776-215a18ebe582 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.584393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.585078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.585364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.585632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.586353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:00:41.586629 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.587669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.587669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:00:41.587669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:00:41.588560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.601148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 776/1496] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:00:41.609034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.619060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfa930-7629-46b2-bf15-0e8767f390b8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.624943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.625673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.625952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.626282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.626968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.628794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.628794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:00:41.628794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:00:41.630049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.671178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 721/1497] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:00:41 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:00:41.673947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.683965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2c8a-0ad7-41bc-8db7-3e0b7cddf329 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.749716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.750343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.750624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.750900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.751688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.752734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.752734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:00:41.752734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:00:41.753988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.776062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 777/1498] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:00:41 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:00:41.780489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.791485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84d4bc-21e5-4f17-8e62-6f500480b8f0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.796499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.797203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.797489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.797765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.798660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.799702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.799702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:00:41.799702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:00:41.800597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.814099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 722/1499] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:00:41.818441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.828232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cce70-16e7-4a14-af2f-7db5bf1b7304 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.833854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.834512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.834788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.835058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.835786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.836796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.836796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:00:41.836796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:00:41.837728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3188a-48b3-4846-a1c7-cd42b29f823a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.851828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 778/1500] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:00:41.856110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:41.866668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d78ce-007c-419b-87d1-1221c04f533a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:41.872362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:41.873291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.873573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.873847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] PATH_INFO: `/v3/projects/8e348e459aee4e6d8d9d7d2234754708/users/901e1ed93e8d4da6a3f72dc9caac85dd/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.889720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e348e459aee4e6d8d9d7d2234754708, user_id=901e1ed93e8d4da6a3f72dc9caac85dd, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:41.890746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:41.890746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:00:41.890746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:00:41.892791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe7508-3a01-4fcd-a35c-f1e4d79b02c5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:41.908915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 723/1501] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:00:41 2022] PUT /identity/v3/projects/8e348e459aee4e6d8d9d7d2234754708/users/901e1ed93e8d4da6a3f72dc9caac85dd/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:00:41.920499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd996b8-e8f7-4b62-ade5-2c8ff2ef76b1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:41.920786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd996b8-e8f7-4b62-ade5-2c8ff2ef76b1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:41.921111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd996b8-e8f7-4b62-ade5-2c8ff2ef76b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:41.952826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ebd996b8-e8f7-4b62-ade5-2c8ff2ef76b1 None None] MFA Rules not processed for user `901e1ed93e8d4da6a3f72dc9caac85dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:00:41.954073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ebd996b8-e8f7-4b62-ade5-2c8ff2ef76b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:42.024481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 779/1502] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:00:42.038724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234199bb-038d-490f-acce-405bc2b23ecb None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.039010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234199bb-038d-490f-acce-405bc2b23ecb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.039282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234199bb-038d-490f-acce-405bc2b23ecb None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.040670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 724/1503] 10.222.0.55 () {54 vars in 963 bytes} [Tue Jun 28 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:00:42.048479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f30-2839-4a6f-b702-5cbc85f77971 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.049081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f30-2839-4a6f-b702-5cbc85f77971 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.049081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f30-2839-4a6f-b702-5cbc85f77971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.072622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-75551f30-2839-4a6f-b702-5cbc85f77971 None None] MFA Rules not processed for user `e6c561ff3ab84f99b81eaccc9d9f2cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:00:42.073861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-75551f30-2839-4a6f-b702-5cbc85f77971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:42.119500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:00:42.128288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 780/1504] 10.222.0.55 () {58 vars in 1060 bytes} [Tue Jun 28 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:00:42.135837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234199bb-038d-490f-acce-405bc2b23ecb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:42.146791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234199bb-038d-490f-acce-405bc2b23ecb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:42.150491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:42.151126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.151406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.151676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.159988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:42.161472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:42.170177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 725/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:00:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:00:42.212492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f50a0-3eba-4cd5-b280-b211f4818937 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.212781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f50a0-3eba-4cd5-b280-b211f4818937 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.213087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f50a0-3eba-4cd5-b280-b211f4818937 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.214403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 781/1506] 10.222.0.55 () {52 vars in 829 bytes} [Tue Jun 28 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:00:42.218116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:42.282463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f468a28-78e6-4696-91d6-1c887d52209f service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:42.287782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:42.288673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.288959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.289264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] PATH_INFO: `/v3/users/901e1ed93e8d4da6a3f72dc9caac85dd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.290536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901e1ed93e8d4da6a3f72dc9caac85dd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:42.291587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:42.291587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:00:42.291587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:00:42.292808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:42.362470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901e1ed93e8d4da6a3f72dc9caac85dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:42.380150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901e1ed93e8d4da6a3f72dc9caac85dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:42.393613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901e1ed93e8d4da6a3f72dc9caac85dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:42.394608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901e1ed93e8d4da6a3f72dc9caac85dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:42.395717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901e1ed93e8d4da6a3f72dc9caac85dd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:42.422740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:00:42.434267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 726/1507] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:00:42 2022] DELETE /identity/v3/users/901e1ed93e8d4da6a3f72dc9caac85dd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:00:42.441099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.441462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.441863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.467835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:00:42.469397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:42.557229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 782/1508] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:00:42.567803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:42.569791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:42.649393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930429c-8c16-4373-9c49-7570409bd7a0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:42.653909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:42.654569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:42.654836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:42.655098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:42.665179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:42.666638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492ecb3-dd5b-415d-b4a3-46818a7a9b29 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:42.676383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 727/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:00:43.096254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:00:43.098579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:00:43.121301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702ec47-7ea7-4c82-a2ed-913ac2c71916 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:00:43.127639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:00:43.128562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:00:43.128857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:00:43.129140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] PATH_INFO: `/v3/projects/8e348e459aee4e6d8d9d7d2234754708` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:00:43.130398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e348e459aee4e6d8d9d7d2234754708)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:00:43.131549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:00:43.131549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:00:43.131549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:00:43.132833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:00:43.212532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e348e459aee4e6d8d9d7d2234754708', 'request_id': 'req-dcfb8b60-f133-49f5-bf98-f319e1c61395'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:43.229013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e348e459aee4e6d8d9d7d2234754708', 'request_id': 'req-dcfb8b60-f133-49f5-bf98-f319e1c61395'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:43.244042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e348e459aee4e6d8d9d7d2234754708', 'request_id': 'req-dcfb8b60-f133-49f5-bf98-f319e1c61395'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:43.261329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] The token cache is being invalidate because project 8e348e459aee4e6d8d9d7d2234754708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:00:43.261617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:43.262240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-dcfb8b60-f133-49f5-bf98-f319e1c61395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:00:43.263927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 783/1510] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:00:43 2022] DELETE /identity/v3/projects/8e348e459aee4e6d8d9d7d2234754708 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:00:52.676943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:00:53.258020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:02:22.687540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:22.688209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:22.688209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:22.690258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 728/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 12:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:22.697225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872937-72d6-4a58-8188-e07a3ceeb132 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:22.697648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872937-72d6-4a58-8188-e07a3ceeb132 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:22.697778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16872937-72d6-4a58-8188-e07a3ceeb132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:22.726282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-16872937-72d6-4a58-8188-e07a3ceeb132 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:22.727508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-16872937-72d6-4a58-8188-e07a3ceeb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:22.792501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 784/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:22.949954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:22.951985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:23.032858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94d394-aebc-4e65-97d6-365e6b304bfa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:23.039679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:23.040344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:23.040623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:23.040892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:23.050525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:23.052278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898a851-5827-4399-891f-efd4d7cfcc4c service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:23.062140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 729/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:23.883740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549016a-eb74-443e-b2b2-d0884915e83c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:23.884375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549016a-eb74-443e-b2b2-d0884915e83c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:23.884375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549016a-eb74-443e-b2b2-d0884915e83c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:23.885742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 785/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:23.891840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb616f0-910a-4745-bfad-10a9c6ce3bec None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:23.892290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb616f0-910a-4745-bfad-10a9c6ce3bec None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:23.892433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb616f0-910a-4745-bfad-10a9c6ce3bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:23.914170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-adb616f0-910a-4745-bfad-10a9c6ce3bec None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:23.915384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-adb616f0-910a-4745-bfad-10a9c6ce3bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:23.924267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 730/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:24.090936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549016a-eb74-443e-b2b2-d0884915e83c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:24.107495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549016a-eb74-443e-b2b2-d0884915e83c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:24.115870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:24.116522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:24.116803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:24.117103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:24.125826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:24.127452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410d8ac-666e-493a-bfee-22bfa8d67dce service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:24.135143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624979-a2e6-48c6-829b-08f1251486d9 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:24.135433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624979-a2e6-48c6-829b-08f1251486d9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:24.135748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7624979-a2e6-48c6-829b-08f1251486d9 None None] PATH_INFO: `/v3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:24.137083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 731/1516] 10.222.0.50 () {54 vars in 925 bytes} [Tue Jun 28 12:02:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:02:24.137671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 786/1517] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:24.143434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b57cf-b45f-4f7a-82c0-22f1cc3c2de6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:24.143720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b57cf-b45f-4f7a-82c0-22f1cc3c2de6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:24.144156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b57cf-b45f-4f7a-82c0-22f1cc3c2de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:24.163074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-190b57cf-b45f-4f7a-82c0-22f1cc3c2de6 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:24.164289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-190b57cf-b45f-4f7a-82c0-22f1cc3c2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:24.233530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 732/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Jun 28 12:02:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:33.300207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b84564-546f-4e65-a275-5f01960f61ce None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:33.301096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b84564-546f-4e65-a275-5f01960f61ce None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:33.301096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b84564-546f-4e65-a275-5f01960f61ce None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:33.303279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 787/1519] 10.222.0.55 () {54 vars in 960 bytes} [Tue Jun 28 12:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:33.309116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31d4f-6ab6-43a1-970b-afe61c0b4c98 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:33.309604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31d4f-6ab6-43a1-970b-afe61c0b4c98 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:33.309707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c31d4f-6ab6-43a1-970b-afe61c0b4c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:33.329410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a2c31d4f-6ab6-43a1-970b-afe61c0b4c98 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:33.330591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a2c31d4f-6ab6-43a1-970b-afe61c0b4c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:33.341658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 733/1520] 10.222.0.55 () {58 vars in 1057 bytes} [Tue Jun 28 12:02:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:33.350163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b84564-546f-4e65-a275-5f01960f61ce None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:33.366789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b84564-546f-4e65-a275-5f01960f61ce None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:33.372432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:33.373098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:33.373383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:33.373667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:33.382192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:33.383999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f594b-ceb5-4f8a-9a86-6fb0d00f6036 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:33.393746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 788/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:35.431021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e380-696d-478a-8610-b5af03dd7b1a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:35.431651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e380-696d-478a-8610-b5af03dd7b1a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:35.431651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5e380-696d-478a-8610-b5af03dd7b1a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:35.433011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 734/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 12:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:35.438125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7065-5e1e-4b01-8eb8-14f5c1c116f9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:35.438585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7065-5e1e-4b01-8eb8-14f5c1c116f9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:35.438681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7065-5e1e-4b01-8eb8-14f5c1c116f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:35.458841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8d1c7065-5e1e-4b01-8eb8-14f5c1c116f9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:35.460062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8d1c7065-5e1e-4b01-8eb8-14f5c1c116f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:35.468740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 789/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 12:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:35.625521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e380-696d-478a-8610-b5af03dd7b1a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:35.640764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5e380-696d-478a-8610-b5af03dd7b1a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:35.647162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:35.647795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:35.648078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:35.648351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:35.657548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:35.659182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8110f-d401-4dcc-84cf-69c744816066 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:35.668894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 735/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:36.502410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadd3c-7fe7-4359-a978-37c7fcb297a3 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:36.503243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadd3c-7fe7-4359-a978-37c7fcb297a3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:36.503243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadd3c-7fe7-4359-a978-37c7fcb297a3 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:36.505480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 790/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 28 12:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:36.510693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a41f-a513-4d47-855a-8334f17694e6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:36.511172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a41f-a513-4d47-855a-8334f17694e6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:36.511269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2a41f-a513-4d47-855a-8334f17694e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:36.531611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-76b2a41f-a513-4d47-855a-8334f17694e6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:36.532822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-76b2a41f-a513-4d47-855a-8334f17694e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:36.543188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 736/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 28 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:36.696609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eadd3c-7fe7-4359-a978-37c7fcb297a3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:36.722656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eadd3c-7fe7-4359-a978-37c7fcb297a3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:36.730875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:36.731526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:36.731808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:36.732088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:36.741207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:36.743009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba258f-eceb-4859-b29a-bb811a15408c service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:36.752957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 791/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:40.692273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:40.692886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:40.692886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:40.694971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 737/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 12:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:40.704951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402b69-66cc-4347-8849-c76e2b2cd043 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:40.705767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402b69-66cc-4347-8849-c76e2b2cd043 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:40.705949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402b69-66cc-4347-8849-c76e2b2cd043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:40.727901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3c402b69-66cc-4347-8849-c76e2b2cd043 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:40.729291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3c402b69-66cc-4347-8849-c76e2b2cd043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:40.738358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 792/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 12:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:40.776859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:40.779922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:40.841559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:02:40.849552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e7db1-684c-493a-9922-e8b92b8f10ae None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:40.854222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:40.855254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:40.855565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:40.855924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:40.870511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:40.871986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8e1e5-1d5a-43c0-b842-0273c83895ff service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:40.881499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 738/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:42.912223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb76-8875-4aee-a5d3-9466c4aa6480 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:42.912785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb76-8875-4aee-a5d3-9466c4aa6480 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:42.912785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcb76-8875-4aee-a5d3-9466c4aa6480 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:42.915809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 793/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 12:02:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:42.924745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ca0a-b0c5-4485-8af9-7ac81db9eb64 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:42.925204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ca0a-b0c5-4485-8af9-7ac81db9eb64 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:42.925347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ca0a-b0c5-4485-8af9-7ac81db9eb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:42.946872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31c8ca0a-b0c5-4485-8af9-7ac81db9eb64 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:42.948098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31c8ca0a-b0c5-4485-8af9-7ac81db9eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:42.958044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 739/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 12:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:42.995297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bcb76-8875-4aee-a5d3-9466c4aa6480 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:43.015673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bcb76-8875-4aee-a5d3-9466c4aa6480 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:43.021380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:43.021955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:43.022228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:43.022494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:43.032943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:43.034430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf31817-8157-4356-a12c-ca7876a208ea service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:43.044632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 794/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:46.502202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:46.502783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:46.502783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:46.504056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 740/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 12:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:46.510371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c568ef-bdf4-4989-ab2f-aca5ed1dfe29 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:46.510803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c568ef-bdf4-4989-ab2f-aca5ed1dfe29 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:46.510966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c568ef-bdf4-4989-ab2f-aca5ed1dfe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:46.521178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=58 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:02:46.535484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-11c568ef-bdf4-4989-ab2f-aca5ed1dfe29 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:46.536716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-11c568ef-bdf4-4989-ab2f-aca5ed1dfe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:46.545531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 795/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:46.862500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:46.865640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:46.940524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287698-03c7-4693-8ff5-bf9784c18031 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:46.945231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:46.946278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:46.946533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:46.946816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:46.958249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:46.959687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1184eab-0ecb-44d4-ba00-b1e4380dc396 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:46.970098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 741/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:49.090033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:49.090569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:49.090569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:49.092011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 796/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 28 12:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:02:49.097952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed8b9-be2f-4011-b0f0-c7d6aa451b6a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:49.098429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed8b9-be2f-4011-b0f0-c7d6aa451b6a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:49.098545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed8b9-be2f-4011-b0f0-c7d6aa451b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:49.126107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-187ed8b9-be2f-4011-b0f0-c7d6aa451b6a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:02:49.127308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-187ed8b9-be2f-4011-b0f0-c7d6aa451b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:49.137551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 742/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 28 12:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:02:49.451833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:02:49.453818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:02:49.467960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151e0a0-aa12-4f3d-af63-1f965feb6ef9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:02:49.472189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:02:49.472879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:02:49.473194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:02:49.473473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:02:49.483504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:02:49.484964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c07d-e265-4170-9a9c-6d04870b61b2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:02:49.494730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 797/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:02:59.120594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=36 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:02:59.490619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=8 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:19.816631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:19.817371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:19.817371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:19.842294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:19.843932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:19.856310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 743/1540] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:19.865926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3f94-4db2-459d-888a-ce2da9600f79 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:19.866587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3f94-4db2-459d-888a-ce2da9600f79 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:19.866587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3f94-4db2-459d-888a-ce2da9600f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:19.897085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cb9c3f94-4db2-459d-888a-ce2da9600f79 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:19.898403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cb9c3f94-4db2-459d-888a-ce2da9600f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:19.911733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 798/1541] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:19.916244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:19.930293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fb754-b518-4a20-850b-5863fcd022b6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:19.941097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:19.941799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:19.942084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:19.942356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:19.943101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:19.943412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:19.944446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:19.944446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:04:19.944446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:04:19.945452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:19.949830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625440b8-2f5e-4752-b74a-7258f05b676e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:19.950127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625440b8-2f5e-4752-b74a-7258f05b676e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:19.950402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625440b8-2f5e-4752-b74a-7258f05b676e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:19.967633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 744/1542] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:19.976864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:19.978256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-625440b8-2f5e-4752-b74a-7258f05b676e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:19.979528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-625440b8-2f5e-4752-b74a-7258f05b676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:19.993047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 799/1543] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:19.996007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b2da2-f771-4c85-ae53-42f747ef65bc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:19.998277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0956d5-703c-49de-977c-68b00601d6cb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:19.998564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0956d5-703c-49de-977c-68b00601d6cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:19.998844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0956d5-703c-49de-977c-68b00601d6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.009968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.010662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.010948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.011227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.011952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.014010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93813cb6-40f8-444c-896e-7ed669113749 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.040479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2c0956d5-703c-49de-977c-68b00601d6cb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.041760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2c0956d5-703c-49de-977c-68b00601d6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.059279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 800/1544] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.060941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.061223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.061496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.075249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 745/1545] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:19 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:20.076705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d87b-bef4-46d6-a5b4-bf57ad0603e1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.077042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d87b-bef4-46d6-a5b4-bf57ad0603e1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.077474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1d87b-bef4-46d6-a5b4-bf57ad0603e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.115599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.118499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.124870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d8e1d87b-bef4-46d6-a5b4-bf57ad0603e1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.126077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d8e1d87b-bef4-46d6-a5b4-bf57ad0603e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.197723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 746/1547] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.198081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 801/1547] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.202374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed80d8-fe24-42d7-9343-02f2732354b4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.203473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.204470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed80d8-fe24-42d7-9343-02f2732354b4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.207955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed80d8-fe24-42d7-9343-02f2732354b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.214188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ea69-fa78-4bc7-a759-7347d471ac02 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.219972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.220657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.220965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.221248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.221954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.223917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7feaa-f039-40c8-8a92-c5301a7cac14 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.228785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e7ed80d8-fe24-42d7-9343-02f2732354b4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.230011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e7ed80d8-fe24-42d7-9343-02f2732354b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.238664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 747/1548] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.240034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed570-0c5f-4125-8d7e-32d0808d49a9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.240358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed570-0c5f-4125-8d7e-32d0808d49a9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.240651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed570-0c5f-4125-8d7e-32d0808d49a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.249001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 802/1549] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.249890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.250173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.250450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.259750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-777ed570-0c5f-4125-8d7e-32d0808d49a9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.260976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-777ed570-0c5f-4125-8d7e-32d0808d49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.267961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.269050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 748/1550] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.269197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.270665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b49fa-d88a-41bc-b257-7e8faccbba73 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.270952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b49fa-d88a-41bc-b257-7e8faccbba73 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.271225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b49fa-d88a-41bc-b257-7e8faccbba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.283290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 803/1551] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.284004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.293859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb904b-664b-44e1-9768-6116c76e89f5 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.295490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ac0b49fa-d88a-41bc-b257-7e8faccbba73 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.296755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac0b49fa-d88a-41bc-b257-7e8faccbba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.298895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.299550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.299825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.300095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.300820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.301154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.302758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f4049-65df-4a77-a74e-22e041eb0d55 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.304956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 749/1552] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.306385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.306680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.306954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.322395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 804/1553] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.322395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.322395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.322395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.329324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.330555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.337129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.338299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.339397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 750/1554] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.340294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.346353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 805/1555] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.347124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.354836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4792e-5a8e-43ca-ae37-b238c7041630 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.357484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1469d-12aa-4d08-be99-59c4f3f0e5e6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.360054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.360728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.361045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.361322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.361843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.362052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.362358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.362522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.362799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.363067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.363808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.363983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeccbec-d427-4eec-bf4c-e94ec006cfcf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.365448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc0daa-4254-4c0b-afb2-7395cd4b1fbe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.376153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 806/1556] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:20.388934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 751/1557] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.389159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.389273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6c0b-2fdb-44d3-8ec7-0a6c750a379a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.389361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.389451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6c0b-2fdb-44d3-8ec7-0a6c750a379a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.389533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.389607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6c0b-2fdb-44d3-8ec7-0a6c750a379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.403516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-78db6c0b-2fdb-44d3-8ec7-0a6c750a379a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.405042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.405210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-78db6c0b-2fdb-44d3-8ec7-0a6c750a379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.406781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.412795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 752/1558] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.416038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a134e-cf8f-455d-b906-4e746639515a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.416532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a134e-cf8f-455d-b906-4e746639515a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.416699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a134e-cf8f-455d-b906-4e746639515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.418651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 807/1559] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.419818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.431918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb71198-f82b-4ef8-9700-f8b5b4448706 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.437151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.437975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.438252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.438465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-479a134e-cf8f-455d-b906-4e746639515a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.438635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.439228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.439834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-479a134e-cf8f-455d-b906-4e746639515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.440245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:20.440245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:04:20.440245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:04:20.441439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a175fa8-eb72-49c9-8d8a-9b5a3d193fa5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.448702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 753/1560] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.449764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a134e-cf8f-455d-b906-4e746639515a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.459420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a134e-cf8f-455d-b906-4e746639515a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.465452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.466117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.466413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.466687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.467419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.467756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.469976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.474805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 808/1561] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.474805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.474805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.474805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.482565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 754/1562] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.487145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.492599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:20.493852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:20.497751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd5428-9544-4527-a66b-a0378c62fb07 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.544862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:20.545682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:20.556304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.557017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.557307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 809/1563] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:20.557433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.557630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.558143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.558320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.560024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.568670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ff603-571a-4163-a569-c1343fffc7b4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.572979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 755/1564] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:20.572979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.573928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.574588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.574888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.575135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.575876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.577880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.581649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91d939-dbd9-43e9-ada3-4b8ea16669ef admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.586836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.587494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.587776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.588052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.588780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.589113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.591008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.604646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 756/1565] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.605509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.608754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 810/1566] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.608754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.618486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128acf-7cb9-45e9-aafc-ed026aacdf0a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.668131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c1ee-eb99-416e-b069-582e52a338e5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.672741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.673443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.673724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.673999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.674488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.674911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.675167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.675273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.675452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.675725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.676447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.676751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.676854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.678558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.688963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 811/1567] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.690536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.696737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 757/1568] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.696737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.703309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674e01-fc97-4427-978e-fc9db9c7ac1f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.705892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3dad-5c00-44bd-959d-a961a8991b45 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.708931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.709641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.709924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.710198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.711055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.711186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.711799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.712068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.712354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.713070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.713205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.715065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.735894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 812/1569] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/users => generated 291 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.736995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.747252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 758/1570] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.748113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.748235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e7501-4746-44f3-93b0-4523223256a8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.822550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.823276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.823621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825cbc6-83ff-4c0e-ae8c-3fedd0c24489 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.824105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.824105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.824549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:20.824862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.826565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.828369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.829302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.829584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.830309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] PATH_INFO: `/v3/projects/e1632e95c1054b4e89203492e13ad3b7/users/d6275dbb33b544dcb91453a858073606/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.841650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 813/1571] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:20.842736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.852753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1632e95c1054b4e89203492e13ad3b7, user_id=d6275dbb33b544dcb91453a858073606, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.854424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bcc25-f99d-4a71-87cc-a6d71f2aa2bc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.855512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.859831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.860785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.861193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.861492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.862249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.864412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.871306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 759/1572] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:20 2022] PUT /identity/v3/projects/e1632e95c1054b4e89203492e13ad3b7/users/d6275dbb33b544dcb91453a858073606/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:20.872385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.886451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 814/1573] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.887292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.897451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7eb54-bc34-4e5a-bd80-37e03806379e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.909591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:20.926214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:20.941284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb54f74-326a-426c-97b9-5c47037cb0c0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:20.946429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.947092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.947370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.947643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.948328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.950301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.956860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:20.957555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:20.957835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:20.958110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:20.958801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:20.960722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:20.976872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 760/1574] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.977856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:20.993333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 815/1575] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:20.993333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.046223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bed1b0-50d0-43c9-8244-d097aed6bdb6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.056670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572eb5d-c6d5-4dbb-9e1c-4519741e77ec admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.061618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.062296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.062558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.062831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.063560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.065195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.075164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 816/1576] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.076026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.085951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a94ee9-8931-40cf-9432-bbc23317734a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.091750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.092353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.092671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.092924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.094020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.096130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.103220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.103880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.104165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.104444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.105213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.107121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f909679-fba0-4bb7-adb7-5561ddd6efef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.118778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 817/1577] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.119963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.134162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373f210-2283-413d-b846-0005c99f500f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.138047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 761/1578] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:20 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.139740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.140102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.140396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.140499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.140664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.140793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.141177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.141896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.143893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.160260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:21.161535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:21.174518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 818/1579] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.175616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.210539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:21.230338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 762/1580] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:21.231407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.242634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0760a-5fd0-446a-b8b7-e698b943ef44 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.256553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fe2b7-cdf5-40ea-84da-58778fc3b3da admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.261925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.262616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.262901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.263179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.263919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.265574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.279075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 819/1581] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.279075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.290288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6f3ad-bf36-40e6-9925-5927570035a4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.295748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.296402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.296678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.296973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.297654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.299552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.300480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.301285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.301597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.301869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.302558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.303615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:21.303615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:04:21.303615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:04:21.305074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.306865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:21.323215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 820/1582] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/users => generated 283 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.324377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.325442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 763/1583] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.326415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.336091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52510d5-680f-4ee3-aebe-f7c85aa93e1f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.337194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0800-5a9d-4bab-bae9-29e41ff2cfa4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.341069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.341721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.341934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.342112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.342270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.342501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.342778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.342975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.343501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.344101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.344951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.345724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.354976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 764/1584] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.355742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.356776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 821/1585] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.357596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.367892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6738b-6a4a-44e2-bb18-683516f4cde9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.368304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d67729-e560-4746-80d5-19637df29ff0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.372418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.373028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.373154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.373461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.373703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.373818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.373966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.374246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.374356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.374924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.376256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.376836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.399079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 822/1586] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.400380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.400930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 765/1587] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:21.401874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.411810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c86e1-083d-497f-9397-9cb59b0427a2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.418429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.419092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.419387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.419649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.420405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.422055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.436778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 766/1588] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.436778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.446253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f29ee-64b1-4219-939f-51e7400237bb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.450738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.451377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.451656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.451935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.452666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.454284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.456833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5e581-d947-4afa-b9a6-be010dda000a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.461154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.461851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.462261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.462538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.467523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 767/1589] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.467523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.472670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.474173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.475149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71910c77-c029-4768-b1d3-c4fbd6c50030 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.480800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.481485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.481770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.482041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.482757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.484356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.485273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 823/1590] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:21.486278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.498035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 768/1591] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.498035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.501330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d26bc0-6b4b-400a-948b-0b4ed1d123ef service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.506758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.507687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.508704 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.508843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.509609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.510428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f391bb-c026-48b5-9998-2408a466c4d0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.511216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.519804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.521012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.521446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.521861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] PATH_INFO: `/v3/projects/9093a24833e6400098728603fb9e8306/users/187b061de2574e6d9a42584042f2c71e/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.526626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 824/1592] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.527579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.537985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a35834-5c78-4bbc-8c39-3358aea70c96 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.543227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.543903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.544181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.544450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.545223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.546874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.548817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9093a24833e6400098728603fb9e8306, user_id=187b061de2574e6d9a42584042f2c71e, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.552018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.558400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 825/1593] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.559479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.570828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30b574-126a-4862-b684-54e44cb254f7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.572017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 769/1594] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:21 2022] PUT /identity/v3/projects/9093a24833e6400098728603fb9e8306/users/187b061de2574e6d9a42584042f2c71e/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:21.572627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.646261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:21.657452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.658170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.658450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.658724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.659453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.660304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f55a50-e60f-4dc1-ac19-2ede5cbc04da admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.661252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.667650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.668884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.668884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.669319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] PATH_INFO: `/v3/projects/c47f35f9ae5e4a55be23a84e3887b915/users/2a54e9db924c440da63c04052fe058e6/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.673004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 826/1595] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.673768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.687775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1133407-8930-4997-91f6-02200cf535aa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.692935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47f35f9ae5e4a55be23a84e3887b915, user_id=2a54e9db924c440da63c04052fe058e6, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.698781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.703886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.704825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.705162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.705498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.706304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.708118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.720066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 770/1596] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:21 2022] PUT /identity/v3/projects/c47f35f9ae5e4a55be23a84e3887b915/users/2a54e9db924c440da63c04052fe058e6/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:21.720066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.720368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 827/1597] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.720368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.732634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c073-e5f2-44a2-8ef8-89114afd2fcc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.758840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:21.798815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.799770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.800452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.800452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] PATH_INFO: `/v3/projects/4f6581f99b314a25a57c25ce087b60f9/users/aaaf6c2b064745648e96bcc3f968f627/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.801996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ef6f-e7b9-4960-8036-92f1f2a98fdf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.809043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.809899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.810188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.810486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.811213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.812898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.817448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6581f99b314a25a57c25ce087b60f9, user_id=aaaf6c2b064745648e96bcc3f968f627, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.818777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:21.818777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:04:21.818777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:04:21.820609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5023-7629-4cc9-a7d3-e6473bdd0333 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.824347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 771/1598] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.825385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.835538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91dc3a-b93f-4235-a107-6154b760349d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.841380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 828/1599] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:21 2022] PUT /identity/v3/projects/4f6581f99b314a25a57c25ce087b60f9/users/aaaf6c2b064745648e96bcc3f968f627/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:21.842708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b0ab-0610-4ae0-940c-97a46c5b4ab5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.842994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b0ab-0610-4ae0-940c-97a46c5b4ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.843640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95b0ab-0610-4ae0-940c-97a46c5b4ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.888843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2f95b0ab-0610-4ae0-940c-97a46c5b4ab5 None None] MFA Rules not processed for user `187b061de2574e6d9a42584042f2c71e`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:21.890290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2f95b0ab-0610-4ae0-940c-97a46c5b4ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:21.923294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.924720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.925343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.925909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.927595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.930908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.946550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 772/1600] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.953203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:21.965834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f679e6d-21f4-46c1-b46c-f4df698ddb6a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:21.972805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:21.973598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.973982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.974359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.974894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 829/1601] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:21.975321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:21.977235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:21.979166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:21.979937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:21.980782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:21.992659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 773/1602] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:21.996109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.008121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.010336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cd0a-2798-4ca0-8d0f-dcfae7765bc1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.010492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.016281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.017268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.017620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.017839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] PATH_INFO: `/v3/projects/6cfd5e85dd174097a281644baa7bfccd/users/ac091d8e38054fa480ce7b9f4bb9b98b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.023221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 830/1603] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.025827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.038743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfd5e85dd174097a281644baa7bfccd, user_id=ac091d8e38054fa480ce7b9f4bb9b98b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.040892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0e22-0546-4d6d-8515-e58b4a98212a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.042642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0070cd7-8fb7-4b4e-8f4d-840f095c0cb4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.045767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.046779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.047130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.047391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] PATH_INFO: `/v3/projects/bc6cfc425cc74fdfaa65a7a3cf60ffb4/users/93cb5dcef86246ada2d7ee4a08b0d733/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.058707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 774/1604] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:21 2022] PUT /identity/v3/projects/6cfd5e85dd174097a281644baa7bfccd/users/ac091d8e38054fa480ce7b9f4bb9b98b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:22.068485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.068485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.068485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.068769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6cfc425cc74fdfaa65a7a3cf60ffb4, user_id=93cb5dcef86246ada2d7ee4a08b0d733, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.069256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.092570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 831/1605] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:22 2022] PUT /identity/v3/projects/bc6cfc425cc74fdfaa65a7a3cf60ffb4/users/93cb5dcef86246ada2d7ee4a08b0d733/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:22.092570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.107793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] MFA Rules not processed for user `aaaf6c2b064745648e96bcc3f968f627`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.109782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.133559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:22.163723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1bb2e-ca10-4cbd-a93c-ac1df5a6782a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.171831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.173078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.173356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.173628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] PATH_INFO: `/v3/projects/73679a9c1e1140dab3237b74ff50d3c8/users/b8d8d0c23f744a7b9ad311581f6ae39f/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.185711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 775/1606] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.186756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.192098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:22.195061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73679a9c1e1140dab3237b74ff50d3c8, user_id=b8d8d0c23f744a7b9ad311581f6ae39f, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.199971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05ef0e-5add-46b9-b6c3-c9e6da850aa8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.201221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef29ac-7165-48e6-8fd7-00bcb1b6b9e9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.206892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.207914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.208256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.208820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] PATH_INFO: `/v3/projects/cb27799cde2e4258a2eedc5bda555c61/users/702dcd49668c4680b5bf3c1c75575ab7/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.220420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 832/1607] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:22 2022] PUT /identity/v3/projects/73679a9c1e1140dab3237b74ff50d3c8/users/b8d8d0c23f744a7b9ad311581f6ae39f/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:22.222534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.223049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.223382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.225193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 833/1608] 10.222.0.55 () {54 vars in 961 bytes} [Tue Jun 28 12:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:22.226186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.229368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.249979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb27799cde2e4258a2eedc5bda555c61, user_id=702dcd49668c4680b5bf3c1c75575ab7, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.252885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576e7f1-742f-49e1-8901-86fad57d28a4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.274858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 776/1609] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:22 2022] PUT /identity/v3/projects/cb27799cde2e4258a2eedc5bda555c61/users/702dcd49668c4680b5bf3c1c75575ab7/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:22.276767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c045-07ed-4e74-8610-3b38b75e6808 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.277271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c045-07ed-4e74-8610-3b38b75e6808 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.278167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31c045-07ed-4e74-8610-3b38b75e6808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.312668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7807b3a-bad2-44b8-b169-4bc8b91c0254 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.323505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5c31c045-07ed-4e74-8610-3b38b75e6808 None None] MFA Rules not processed for user `ac091d8e38054fa480ce7b9f4bb9b98b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.326129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c31c045-07ed-4e74-8610-3b38b75e6808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.381554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.382406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.382750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.383133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.430259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 777/1610] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.430259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a6bb8-e7b9-4137-b527-9a0c7b3703ba None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.430259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a6bb8-e7b9-4137-b527-9a0c7b3703ba None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.430259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a6bb8-e7b9-4137-b527-9a0c7b3703ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.468547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a44a6bb8-e7b9-4137-b527-9a0c7b3703ba None None] MFA Rules not processed for user `93cb5dcef86246ada2d7ee4a08b0d733`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.470973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a44a6bb8-e7b9-4137-b527-9a0c7b3703ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.476858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.478348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.494019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 834/1611] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:22.496749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.569369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da7db-4b6a-43f5-8569-2f4b1eb5f131 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.574465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.575453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 778/1612] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.575831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.576265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.576677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.576955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96028c-d74d-4b66-b944-fa243d3787f0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.577251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96028c-d74d-4b66-b944-fa243d3787f0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.577581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96028c-d74d-4b66-b944-fa243d3787f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.591244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:22.609009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4c96028c-d74d-4b66-b944-fa243d3787f0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.609009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c96028c-d74d-4b66-b944-fa243d3787f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.616807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 779/1613] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.618695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.619140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.619545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.647200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.648709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.657583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:22.675688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.678024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a94349-8093-4eb6-8515-03ac98bda26d service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.690328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 835/1614] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:22.692063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.692500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.692956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.712068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:22.714053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:22.724701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 836/1615] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.725839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.739701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09233635-4af2-4162-96f9-5bbf887c195b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.745675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.746372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.746655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.746927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.747631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.749633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.788399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 780/1616] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:22.789530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.803018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de347c4-3e7a-4e40-b030-0fc9d5a2f333 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.804953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 837/1617] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:22.811228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.890765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.891364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.891568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.892673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.893315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.895684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.925677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55ca56-0f25-45bc-a89d-a7bb6fedbf90 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:22.935014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:22.935698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:22.935979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:22.936255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:22.950774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 781/1618] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:22 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:22.952611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:22.959559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:22.961579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:22.984814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 838/1619] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:22.985730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.063031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2913ee5-a338-4182-9d84-569ce38cd3e3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.069112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.070098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.070538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.071023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.124504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff643c-8e95-4204-b473-b0424599add1 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.155747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:23.168173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:23.227562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.231867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.241716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.248705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.249480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.249913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.258298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 782/1620] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:23.259155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.289420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c570139-e589-40ef-8f4d-6f0749484b73 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.299282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.300020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.300303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.301073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.302128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.305257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.335372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 783/1621] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:23 2022] POST /identity/v3/users => generated 289 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:23.336423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.348775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48325d-9ea4-4633-a129-ddcb2d7aca57 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.359685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.360140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.360428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.360698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.361425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.364549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.365854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.368820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.387571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 839/1622] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:23.389102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.410985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e267-2bc8-481d-9473-20f65820903b service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.431990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.432952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.433377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.433657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.434616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.436230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.440632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 784/1623] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:23 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:23.441569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.467959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 840/1624] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:23.468515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ece0c-52c2-4e1c-aece-e70642db6f13 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.469978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.478151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.478853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.479132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.479405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.480141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.481820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.493379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01c487-0fd7-4a76-b411-e661326dcc80 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.514969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 785/1625] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:23.519868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.524202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.525650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.525932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.526207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] PATH_INFO: `/v3/projects/a63fc42773a24f259a78eaf989ea22c4/users/5ea3b0aab3734e0b9ffde8ef6705ac30/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.606853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8b9d-3413-4af4-86dc-30c1582d5d0d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.606853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.606853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.606853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.623343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] PATH_INFO: `/v3/projects/f877a302f0f44c77a0dd65ef2487bca7/users/f470c32b1525402f8a177e645ad4f58a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.624231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63fc42773a24f259a78eaf989ea22c4, user_id=5ea3b0aab3734e0b9ffde8ef6705ac30, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.624231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.629436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 841/1626] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:23 2022] PUT /identity/v3/projects/a63fc42773a24f259a78eaf989ea22c4/users/5ea3b0aab3734e0b9ffde8ef6705ac30/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:23.632057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.648993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f877a302f0f44c77a0dd65ef2487bca7, user_id=f470c32b1525402f8a177e645ad4f58a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.657304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.676166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 786/1627] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:23 2022] PUT /identity/v3/projects/f877a302f0f44c77a0dd65ef2487bca7/users/f470c32b1525402f8a177e645ad4f58a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:23.681555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.811460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e2ef-5f2f-4434-a2a0-2c50ee8d7ab1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.817147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.818120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.818555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.818953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.820118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.821946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.841722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 787/1628] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:04:23.846192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.859953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb381bc1-cec4-4635-9f8e-ed0f363d38fc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.869844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.872814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81de6-e829-4151-9980-69f400ce6fb7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.877458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.878050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.878277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:23.878963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.879241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.879508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.880007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] PATH_INFO: `/v3/projects/f877a302f0f44c77a0dd65ef2487bca7/users/f470c32b1525402f8a177e645ad4f58a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.880231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.881992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.883189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f877a302f0f44c77a0dd65ef2487bca7, user_id=f470c32b1525402f8a177e645ad4f58a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:23.888181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6913f2a-628b-42c6-aa2e-5060408cbb37 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:23.900521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 842/1629] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:23.904497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 788/1630] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:23 2022] PUT /identity/v3/projects/f877a302f0f44c77a0dd65ef2487bca7/users/f470c32b1525402f8a177e645ad4f58a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:23.905042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:23.920381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c5b7-0993-4829-8aa6-a0f07175971c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:23.922111 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9b95-780d-40d1-931b-ba0b23904f75 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:23.937760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c5b7-0993-4829-8aa6-a0f07175971c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:23.939426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c5b7-0993-4829-8aa6-a0f07175971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:23.982593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:24.011169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-40b0c5b7-0993-4829-8aa6-a0f07175971c None None] MFA Rules not processed for user `f470c32b1525402f8a177e645ad4f58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:24.012485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-40b0c5b7-0993-4829-8aa6-a0f07175971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:24.080050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:24.081707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.082018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.082575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] PATH_INFO: `/v3/projects/a63fc42773a24f259a78eaf989ea22c4/users/5ea3b0aab3734e0b9ffde8ef6705ac30/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.086430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63fc42773a24f259a78eaf989ea22c4, user_id=5ea3b0aab3734e0b9ffde8ef6705ac30, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:24.089428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eb6d6-1e02-416a-8493-0bbea72f924f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:24.100420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:24.110413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 843/1631] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:23 2022] PUT /identity/v3/projects/a63fc42773a24f259a78eaf989ea22c4/users/5ea3b0aab3734e0b9ffde8ef6705ac30/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:24.117863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372452-df91-4293-88ce-308b85d67a3e None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.117863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372452-df91-4293-88ce-308b85d67a3e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.117863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372452-df91-4293-88ce-308b85d67a3e None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.117863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 844/1632] 10.222.0.55 () {54 vars in 912 bytes} [Tue Jun 28 12:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:24.125828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.127859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.128932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.171467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] MFA Rules not processed for user `5ea3b0aab3734e0b9ffde8ef6705ac30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:24.171467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:24.174503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 789/1633] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:24.176596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.176893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.177279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.214113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:24.218838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:24.297691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 845/1634] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:24.299028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:24.357768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 790/1635] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:24.358891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:24.391996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df6fbd-7e63-4afc-a2d9-001db9b733b9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:24.397227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:24.398010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.398660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.399015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.465991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5901d6-7f0d-47ef-94bf-009f6f8e8279 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:24.474765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:24.475584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.475774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.476084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.504935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:24.507329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:24.508950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:24.531057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:24.532518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:24.548556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 791/1636] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:24.555624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 846/1637] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:24.622602 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:24.634181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9faf42-5f7a-430f-9d78-a612b0e15adc service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:24.643723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:24.644405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.644688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.645628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:24.665359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:24.669871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec042c3-0285-4597-905c-1002d238e3ba service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:24.693504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 792/1638] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:24.841186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:24.852558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6d0fe-f895-45e6-bde2-cc95fc54e89f service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:24.868158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:24.868840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:24.869151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:24.869426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:25.021075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:25.023421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f681bf-0645-4a46-aba8-386bd7838e10 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:25.040131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 847/1639] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:25.903649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466db3-05c7-4252-b8b4-f9c5daeea8a5 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:25.904835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466db3-05c7-4252-b8b4-f9c5daeea8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:25.905337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37466db3-05c7-4252-b8b4-f9c5daeea8a5 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:25.907326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 793/1640] 10.222.0.55 () {54 vars in 961 bytes} [Tue Jun 28 12:04:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:25.916265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477af8e-eddd-4650-83a9-5b5cd34a15e5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:25.916987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477af8e-eddd-4650-83a9-5b5cd34a15e5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:25.917379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477af8e-eddd-4650-83a9-5b5cd34a15e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:25.948485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c477af8e-eddd-4650-83a9-5b5cd34a15e5 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:25.952451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c477af8e-eddd-4650-83a9-5b5cd34a15e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:26.011821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 848/1641] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Jun 28 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:26.019236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1eed-7f27-4bcd-99bc-04d82f5bb67a None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:26.019963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1eed-7f27-4bcd-99bc-04d82f5bb67a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:26.020429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1eed-7f27-4bcd-99bc-04d82f5bb67a None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:26.022015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 794/1642] 10.222.0.55 () {54 vars in 912 bytes} [Tue Jun 28 12:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:26.028943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:26.029903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:26.030545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:26.055103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:26.056767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:26.093422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 849/1643] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:26.109574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f1eed-7f27-4bcd-99bc-04d82f5bb67a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:26.123887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f1eed-7f27-4bcd-99bc-04d82f5bb67a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:26.134105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:26.134784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:26.136227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:26.136518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:26.156361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:26.157923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374cbaf-078b-4a4c-8914-134df11252f2 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:26.166867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:26.212317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 795/1644] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:26.272134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51504b24-9bf5-4511-9eb0-441a9b930581 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:26.279896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:26.280985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:26.281417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:26.281644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] PATH_INFO: `/v3/users/ac091d8e38054fa480ce7b9f4bb9b98b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:26.283264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac091d8e38054fa480ce7b9f4bb9b98b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:26.284242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: WARNING py.warnings [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:26.284242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: warnings.warn(msg) Jun 28 12:04:26.284242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: Jun 28 12:04:26.285555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:26.438075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac091d8e38054fa480ce7b9f4bb9b98b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:26.461345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac091d8e38054fa480ce7b9f4bb9b98b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:26.479724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac091d8e38054fa480ce7b9f4bb9b98b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:26.480694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac091d8e38054fa480ce7b9f4bb9b98b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:26.481397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac091d8e38054fa480ce7b9f4bb9b98b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:26.503228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:26.515862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 850/1645] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:26 2022] DELETE /identity/v3/users/ac091d8e38054fa480ce7b9f4bb9b98b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:26.517398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:26.517757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:26.518159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:26.562985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:26.567980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:26.692637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:26.775555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 796/1646] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:26.841350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:26.843672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:27.052399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e01582-7126-47c6-a59a-61424eed6cbf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:27.070343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:27.071867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:27.072112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:27.072679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:27.094349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:27.095834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:27.117765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 851/1647] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:29.170931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:29.172736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:29.204983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5994-7b9b-4a7a-bf24-2b58976ded25 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:29.221965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:29.225237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:29.225765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:29.226100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] PATH_INFO: `/v3/projects/6cfd5e85dd174097a281644baa7bfccd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:29.232024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfd5e85dd174097a281644baa7bfccd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:29.234636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: WARNING py.warnings [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 28 12:04:29.234636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: warnings.warn(msg) Jun 28 12:04:29.234636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: Jun 28 12:04:29.236324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:29.440310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfd5e85dd174097a281644baa7bfccd', 'request_id': 'req-d563b20c-4654-472d-b818-17d419731722'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:29.481476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfd5e85dd174097a281644baa7bfccd', 'request_id': 'req-d563b20c-4654-472d-b818-17d419731722'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:29.517549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfd5e85dd174097a281644baa7bfccd', 'request_id': 'req-d563b20c-4654-472d-b818-17d419731722'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:29.566952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] The token cache is being invalidate because project 6cfd5e85dd174097a281644baa7bfccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:29.567330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:29.571568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d563b20c-4654-472d-b818-17d419731722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:29.574444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 797/1648] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:29 2022] DELETE /identity/v3/projects/6cfd5e85dd174097a281644baa7bfccd => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:29.579187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:29.594244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbafb55-be89-488b-9a37-c1189508f99d service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:29.817942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:29.822483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:29.822781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:29.825201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] PATH_INFO: `/v3/users/5ea3b0aab3734e0b9ffde8ef6705ac30` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:29.829257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea3b0aab3734e0b9ffde8ef6705ac30)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:29.837321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:30.001557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea3b0aab3734e0b9ffde8ef6705ac30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:30.021108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea3b0aab3734e0b9ffde8ef6705ac30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:30.034226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea3b0aab3734e0b9ffde8ef6705ac30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:30.035006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea3b0aab3734e0b9ffde8ef6705ac30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:30.035661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15ef5d3a-fc94-45b9-8517-1e93ee777ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea3b0aab3734e0b9ffde8ef6705ac30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:30.088702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 852/1649] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:29 2022] DELETE /identity/v3/users/5ea3b0aab3734e0b9ffde8ef6705ac30 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:30.831209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:30.832219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:30.832661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:30.885052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] MFA Rules not processed for user `d6275dbb33b544dcb91453a858073606`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:30.886373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:30.975637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 798/1650] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:30.988825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:30.989432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:30.989574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:31.020874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] MFA Rules not processed for user `d6275dbb33b544dcb91453a858073606`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:31.023797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:31.048290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 853/1651] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:31.061588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:31.063162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:31.176344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:31.217494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159de2a2-902a-487d-ad3c-7f5b9e7d04fb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:31.225834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:31.226900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:31.227436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:31.227913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:31.238980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:31.242092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d542-5de6-4407-920d-94744e2d5cf5 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:31.276404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 799/1652] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:31.991918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:31.993604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:32.028000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:32.175441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186773-0478-4a59-a9c7-7417c8550ae1 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.176009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186773-0478-4a59-a9c7-7417c8550ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.176009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33186773-0478-4a59-a9c7-7417c8550ae1 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.179722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 800/1653] 10.222.0.55 () {54 vars in 912 bytes} [Tue Jun 28 12:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 12:04:32.200695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890f79-5445-4d54-a11c-c3b7fb5f75be None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.201209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890f79-5445-4d54-a11c-c3b7fb5f75be None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.201907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890f79-5445-4d54-a11c-c3b7fb5f75be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.246377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef56c4-7e80-468d-b5fd-5d0501d59f42 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:32.265351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:32.266650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.267293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.267769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] PATH_INFO: `/v3/projects/a63fc42773a24f259a78eaf989ea22c4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.269341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63fc42773a24f259a78eaf989ea22c4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:32.271502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:32.329397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-89890f79-5445-4d54-a11c-c3b7fb5f75be None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:32.330748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-89890f79-5445-4d54-a11c-c3b7fb5f75be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:32.389373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63fc42773a24f259a78eaf989ea22c4', 'request_id': 'req-638e20b8-6eb3-44ab-bef0-a98c0676f596'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:32.414297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63fc42773a24f259a78eaf989ea22c4', 'request_id': 'req-638e20b8-6eb3-44ab-bef0-a98c0676f596'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:32.431346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63fc42773a24f259a78eaf989ea22c4', 'request_id': 'req-638e20b8-6eb3-44ab-bef0-a98c0676f596'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:32.462446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] The token cache is being invalidate because project a63fc42773a24f259a78eaf989ea22c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:32.463274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:32.464762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-638e20b8-6eb3-44ab-bef0-a98c0676f596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:32.469187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 854/1654] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:31 2022] DELETE /identity/v3/projects/a63fc42773a24f259a78eaf989ea22c4 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:32.477181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.477620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.477994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.502161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 801/1655] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:04:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:32.532078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:32.533319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:32.694100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 855/1656] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:32.702304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.702964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.704365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.727046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:32.729123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:32.747086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:32.747392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:32.781165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 802/1657] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:32.786049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:32.837520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab68664-0b2e-44fb-bb87-a212d4cd51f1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:32.848182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:32.877754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:32.880051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.880051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.882888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.883652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:32.883994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:32.886436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:32.893807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e15b6-3037-401b-9384-fbfdaaf7c6f8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:32.907432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:32.908127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.908410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.908696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.929577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:32.929577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:32.930080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 803/1658] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:32.932187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:32.938949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 856/1659] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:32.960684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02514851-8cfe-460d-9edb-7336e4c35332 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:32.975367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:32.976362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:32.977387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:32.977951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:32.978800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:32.980921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.020924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 804/1660] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:32 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:33.025015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.046189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1be57-43ea-4d3e-9e2b-accade2f84ff service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:33.232484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:33.233306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.233722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.233902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.234575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:33.236562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.298363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 857/1661] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:33 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:33.302721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.318660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47eebe-60dd-489b-9815-57106392cff8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:33.329973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:33.333968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.334443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.335385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.336662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:33.340577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.366683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 805/1662] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:33.370255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.383067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef56b2-411f-4cb5-86b5-5de8ca504773 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:33.398571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:33.399259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.399543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.399818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.400720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:33.402457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b76a-0a47-4d5f-a7cb-b64e8fe0d67d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.424492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 858/1663] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:33.428657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.454155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e92a3-abb2-42c0-a2d0-79d82fefd037 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:33.482916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:33.483848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.484126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.484398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] PATH_INFO: `/v3/projects/fdad1c8493db416c8da8d75e35252313/users/5f477bdfa18d4452ab9c6904157b76fc/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.529464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdad1c8493db416c8da8d75e35252313, user_id=5f477bdfa18d4452ab9c6904157b76fc, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:33.532356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.564779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 806/1664] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:33 2022] PUT /identity/v3/projects/fdad1c8493db416c8da8d75e35252313/users/5f477bdfa18d4452ab9c6904157b76fc/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:33.575428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.575709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.575992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.632065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] MFA Rules not processed for user `5f477bdfa18d4452ab9c6904157b76fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:33.633455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:33.791025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 859/1665] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:33.795473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.877025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed08d-99b6-4b6a-98ce-f8c19123e53f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:33.882544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:33.883288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:33.883566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:33.883838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:33.884533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:33.886818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:33.892776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:33.928122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 807/1666] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:33 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:33.932458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:33.947141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110436c0-b4d6-456f-93c1-82b0d7d179d8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.020415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:34.061844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.064343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.064870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.065285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.066538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.068429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.115401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 860/1667] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:33 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:34.119098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.137376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8504c89-ea2a-47cf-bd10-f3b8fe6d7b4a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.143658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.144780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.145216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.145617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.146684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.149799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.180116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 808/1668] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:34.186174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.203095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a3a9-9ffc-457b-85d2-950158bb26c8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.215243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.215983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.216266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.216690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.217465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.220772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f23df-05f8-4007-8378-733fe86442f2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.239238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 861/1669] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:34.243933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.260980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466d267-72a2-4eca-8adc-b20fe070324c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.266818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.268659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.269176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.269822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] PATH_INFO: `/v3/projects/619bd4827d834a3996fbcdd43777dd10/users/aa1e9c24db0c4ca48f38308a36d7b68b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.299297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=619bd4827d834a3996fbcdd43777dd10, user_id=aa1e9c24db0c4ca48f38308a36d7b68b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.306634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc6d57-bcaa-42be-b580-d7ea37ae7d7d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.364161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 809/1670] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:34 2022] PUT /identity/v3/projects/619bd4827d834a3996fbcdd43777dd10/users/aa1e9c24db0c4ca48f38308a36d7b68b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:34.365522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.365808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.366085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.437534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] MFA Rules not processed for user `aa1e9c24db0c4ca48f38308a36d7b68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:34.442698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:34.529634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cb32e-ac8f-4ab2-9a8b-c5f9707e41ad None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.530138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cb32e-ac8f-4ab2-9a8b-c5f9707e41ad None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.530507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cb32e-ac8f-4ab2-9a8b-c5f9707e41ad None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.532859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 810/1671] 10.222.0.55 () {54 vars in 960 bytes} [Tue Jun 28 12:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:34.539245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.539645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.539991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.582762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:34.587875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:34.635337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 862/1672] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:34.640119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.706645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 811/1673] 10.222.0.55 () {58 vars in 1057 bytes} [Tue Jun 28 12:04:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:34.715696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.734291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19759-3965-42f5-92bc-9d1a6a90d3d2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.741128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.742759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.743043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.743318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.745253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:34.756279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9b88-fb97-4cd3-8d46-cf43d9f1a82f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.764076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.764782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.765151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.765584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.766765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.767612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.767916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.770275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:34.787149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 812/1674] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:34.821935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 863/1675] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:34 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:34.825742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:34.850348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b9980-1eaf-406d-a1de-c113f51f8915 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:34.976958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:34.977695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:34.977981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:34.978897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:34.979673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:34.986090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:35.024968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 813/1676] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:34 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:35.029046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:35.044744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6c381-9db2-4ba0-b5b6-707906fc7470 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:35.064786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:35.066076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:35.067102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:35.067487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:35.068436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:35.075990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:35.116037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 864/1677] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:35.117537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:35.134671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951a2e3-507d-4e38-8c80-176ede3433cb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:35.151269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:35.152215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:35.152937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:35.152937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] PATH_INFO: `/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf/users/9905c4cf65f244a8b289d1aaf0340fd1/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:35.203842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dd9fb71448441b9189a0f2b6d62eaf, user_id=9905c4cf65f244a8b289d1aaf0340fd1, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:35.206907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:35.242299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:35.254739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 814/1678] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:35 2022] PUT /identity/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf/users/9905c4cf65f244a8b289d1aaf0340fd1/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:35.256005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:35.268806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa258fb-a240-4c41-8876-8ca6b486034c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:35.343388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:35.399106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:35.400482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:35.400836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:35.401392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:35.402643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:35.405049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143eee4a-4874-4e07-9504-81a93ffc0dbf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:35.428992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 865/1679] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:35.441227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:35.457832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c65fd1-a10a-453b-9f50-5969185ca456 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:35.477108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:35.478584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:35.479150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:35.479922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] PATH_INFO: `/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf/users/9905c4cf65f244a8b289d1aaf0340fd1/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:35.482609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dd9fb71448441b9189a0f2b6d62eaf, user_id=9905c4cf65f244a8b289d1aaf0340fd1, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:35.499859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:35.519191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 815/1680] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:35 2022] PUT /identity/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf/users/9905c4cf65f244a8b289d1aaf0340fd1/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:35.532472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:35.533144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:35.533696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:35.584637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] MFA Rules not processed for user `9905c4cf65f244a8b289d1aaf0340fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:35.586447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:35.828143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 866/1681] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:35.836847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:35.839383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:36.048000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b285a0-a6c8-467a-8d22-4803a394a844 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:36.062880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:36.063758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:36.064139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:36.064563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:36.088067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:36.094562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f65a1c-0e8b-4f87-9401-70ad30f91723 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:36.132371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 816/1682] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:04:36.217211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:36.230659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f754c62-3bfc-45a3-a471-46e2acaada45 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:36.253628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:36.254419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:36.254610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:36.254881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:36.394725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:36.402280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc031844-625f-40d4-8028-500c82c664c2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:36.431747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 867/1683] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:37.367103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0485-f4a4-4fc4-92a2-77ab2a1deb65 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:37.367573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0485-f4a4-4fc4-92a2-77ab2a1deb65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:37.367680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0485-f4a4-4fc4-92a2-77ab2a1deb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:37.438323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6deb0485-f4a4-4fc4-92a2-77ab2a1deb65 None None] MFA Rules not processed for user `2a54e9db924c440da63c04052fe058e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:37.439658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6deb0485-f4a4-4fc4-92a2-77ab2a1deb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:37.598242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:37.599062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:37.599475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:37.618959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 817/1684] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:37.630837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:37.631338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:37.631755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:37.671341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] MFA Rules not processed for user `702dcd49668c4680b5bf3c1c75575ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:37.672618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:37.678445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] MFA Rules not processed for user `2a54e9db924c440da63c04052fe058e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:37.680174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:37.718730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 818/1685] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:37.731707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:37.739123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:37.863910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:37.920239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 868/1686] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:37.943068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:37.945964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:38.065910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0702d-a512-4ef7-8c30-e5939e07e2bd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:38.079088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:38.079782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:38.080066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:38.080879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:38.101955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:38.103621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33071c08-4374-4e62-9c68-a1cd731e41a5 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:38.107763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:38.119186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 819/1687] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:38.195474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dfc2d-b148-4577-9068-b84d7c99dad8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:38.207018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:38.207860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:38.208209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:38.208564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:38.240200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:38.242356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:38.307202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 869/1688] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:38.389087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c485-5837-42da-a222-fa66476f028f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:38.390198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c485-5837-42da-a222-fa66476f028f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:38.390686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5c485-5837-42da-a222-fa66476f028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:38.460509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2fa5c485-5837-42da-a222-fa66476f028f None None] MFA Rules not processed for user `b8d8d0c23f744a7b9ad311581f6ae39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:38.461817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2fa5c485-5837-42da-a222-fa66476f028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:38.595816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 820/1689] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:38.611889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:38.626466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61639589-2a2a-4247-abce-f2c286b3ab27 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:38.634251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:38.635168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:38.636004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:38.636606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:38.648394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:38.650434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c674a-c679-4fe6-b2c8-47cd319f4bfb service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:38.663907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 870/1690] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:04:39.109298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:39.110549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:39.110703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:39.112509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 821/1691] 10.222.0.55 () {54 vars in 912 bytes} [Tue Jun 28 12:04:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:39.119706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131e63-eadb-46f5-9be2-9b1e2f755983 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:39.120066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131e63-eadb-46f5-9be2-9b1e2f755983 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:39.120708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131e63-eadb-46f5-9be2-9b1e2f755983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:39.183982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c3131e63-eadb-46f5-9be2-9b1e2f755983 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:39.185747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c3131e63-eadb-46f5-9be2-9b1e2f755983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:39.340491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 871/1692] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:04:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:39.354238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:39.356630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:39.402340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082fe05-5c47-4046-840b-e52244dcdc54 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:39.408594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:39.409927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:39.410431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:39.411484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:39.436452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:39.438341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:39.455215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 822/1693] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:40.369748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548ece-c9ca-4822-954f-20bc1e8d15a5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:40.370677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548ece-c9ca-4822-954f-20bc1e8d15a5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:40.371117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548ece-c9ca-4822-954f-20bc1e8d15a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:40.438233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-71548ece-c9ca-4822-954f-20bc1e8d15a5 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:40.449571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-71548ece-c9ca-4822-954f-20bc1e8d15a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:40.672246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 872/1694] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:40.940775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:40.942323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:41.096541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150591-5c60-4e9d-9de4-9e2c0ed0b593 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:41.104468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:41.105978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:41.106474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:41.106980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:41.120195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:41.135386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6012f-5b18-4af2-956c-b50789329728 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:41.150900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 823/1695] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:04:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 28 12:04:42.471806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:42.472351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:42.473211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:42.544297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] MFA Rules not processed for user `702dcd49668c4680b5bf3c1c75575ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:42.556374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:42.621173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 873/1696] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:42.889748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c343-18c3-45b1-9f8d-ed0a7b3fe954 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:42.890690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c343-18c3-45b1-9f8d-ed0a7b3fe954 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:42.891187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c343-18c3-45b1-9f8d-ed0a7b3fe954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:42.917190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5e87c343-18c3-45b1-9f8d-ed0a7b3fe954 None None] MFA Rules not processed for user `2a54e9db924c440da63c04052fe058e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:42.919264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5e87c343-18c3-45b1-9f8d-ed0a7b3fe954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:42.929493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 824/1697] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:43.247012 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:43.249336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:43.256947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:43.534766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6098bc3-4f4e-4d4f-a350-9a8bcdacad3f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:43.545183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:43.546406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:43.546858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:43.549514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] PATH_INFO: `/v3/users/187b061de2574e6d9a42584042f2c71e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:43.551082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187b061de2574e6d9a42584042f2c71e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:43.554849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:43.715940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187b061de2574e6d9a42584042f2c71e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:43.743167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187b061de2574e6d9a42584042f2c71e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:43.788291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187b061de2574e6d9a42584042f2c71e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:43.794808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187b061de2574e6d9a42584042f2c71e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:43.798555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187b061de2574e6d9a42584042f2c71e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:43.903506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 874/1698] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:43 2022] DELETE /identity/v3/users/187b061de2574e6d9a42584042f2c71e => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:43.908081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:43.908621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:43.908702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:43.974772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:43.978052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:44.246228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:44.297136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 825/1699] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:44.444701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:44.447062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:44.561871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:44.563974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:44.634007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658d2b4-a414-4d40-ae20-212ebce68de0 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:44.692110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:44.693361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:44.693871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:44.694468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] PATH_INFO: `/v3/users/5f477bdfa18d4452ab9c6904157b76fc` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:44.697992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f477bdfa18d4452ab9c6904157b76fc)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:44.704570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:44.826261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcff2b-e3ea-4580-a164-2178c21eb252 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:44.832152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:44.832969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:44.833302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:44.833635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:44.892431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:44.894303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c15cd-163d-42c2-9237-69b93f8154b9 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:44.940428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f477bdfa18d4452ab9c6904157b76fc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:44.964326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 875/1700] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:44.971308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f477bdfa18d4452ab9c6904157b76fc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:44.999705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f477bdfa18d4452ab9c6904157b76fc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:45.001432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f477bdfa18d4452ab9c6904157b76fc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:45.005899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f477bdfa18d4452ab9c6904157b76fc'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:45.085304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 826/1701] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:44 2022] DELETE /identity/v3/users/5f477bdfa18d4452ab9c6904157b76fc => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:45.090148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318b22-6817-4036-846d-b7cf9810d060 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:45.090491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318b22-6817-4036-846d-b7cf9810d060 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:45.090794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318b22-6817-4036-846d-b7cf9810d060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:45.183148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-08318b22-6817-4036-846d-b7cf9810d060 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:45.187219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-08318b22-6817-4036-846d-b7cf9810d060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:45.314239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:45.462511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 876/1702] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:45.488474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:45.490040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:45.775185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:45.775762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:45.776188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:45.813308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645e63f-2777-4dc0-9173-6411c9ddc739 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:45.840154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:45.847165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:45.853274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:45.854549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:45.898737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:45.900500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:45.971904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 827/1703] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:46.157500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:46.159142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:46.200943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358fbb2-4dca-4a96-ba93-9a366809a1d0 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:46.228712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:46.229810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:46.230212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:46.230712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] PATH_INFO: `/v3/projects/9093a24833e6400098728603fb9e8306` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:46.234331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9093a24833e6400098728603fb9e8306)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:46.236692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:46.274957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:46.278858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:46.307549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 877/1704] 10.222.0.50 () {58 vars in 1015 bytes} [Tue Jun 28 12:04:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:46.382803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:46.387552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:46.393608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9093a24833e6400098728603fb9e8306', 'request_id': 'req-23b79abd-5ce1-4459-b87c-7f048e25306e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:46.408492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe9545-e3ff-4cbd-ba66-c06f23df2341 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:46.431682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9093a24833e6400098728603fb9e8306', 'request_id': 'req-23b79abd-5ce1-4459-b87c-7f048e25306e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:46.457994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9093a24833e6400098728603fb9e8306', 'request_id': 'req-23b79abd-5ce1-4459-b87c-7f048e25306e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:46.480095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:46.512215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] The token cache is being invalidate because project 9093a24833e6400098728603fb9e8306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:46.512502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:46.517272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-23b79abd-5ce1-4459-b87c-7f048e25306e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:46.534598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 828/1705] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:46 2022] DELETE /identity/v3/projects/9093a24833e6400098728603fb9e8306 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:46.542056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:46.543189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:46.543482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:46.614018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:46.618866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:46.634662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:46.635823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:46.635823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:46.636126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:46.653952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:46.685206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:46.696133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302f40-9df5-478c-886d-439972b479bc service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:46.733774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 878/1706] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:04:46.902258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 829/1707] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:46.906422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:46.906740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:46.907017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:47.048422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:47.060127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:47.096394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 879/1708] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:47.097786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:47.134451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa6af5-212b-48c8-b213-5e3cf6516617 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:47.177595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:47.182849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:47.186668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:47.187288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:47.190844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:47.199584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:47.211214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:47.270682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 830/1709] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:47.272064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:47.286020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c061d7a-a815-4d5c-829e-b7ac77adf631 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:47.311214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:47.312732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:47.315640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:47.316828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:47.321926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:47.335643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:47.469608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 880/1710] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:47 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:47.472162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:47.491140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699185-644c-4d14-ae21-8103ee2008e3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:47.669831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:47.671087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:47.672617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:47.672929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:47.674952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:47.679810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:47.791705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 831/1711] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:47 2022] POST /identity/v3/users => generated 291 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:47.795235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:47.833599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429a75a-8750-4345-be1e-859f45b57892 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:47.847311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:47.860829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:47.866512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:47.867189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:47.867503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:47.888266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:47.959404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 881/1712] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:47.963527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:47.981841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba200f-0952-4dcc-b897-5784d863540c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:48.012659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:48.013468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:48.013697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:48.031738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:48.037046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:48.042004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c71ee-8bbb-404e-b025-aa9aece902de admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:48.111620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 832/1713] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:04:48.114453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:48.137769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e90822-9c04-45bc-829f-bb1d74ddbc99 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:48.167068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:48.168137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:48.168289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:48.168888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] PATH_INFO: `/v3/projects/c5679f30975246b19d849ff3aca53b1e/users/82ab2a3d306648259020716d3e7650ef/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:48.249335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5679f30975246b19d849ff3aca53b1e, user_id=82ab2a3d306648259020716d3e7650ef, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:48.255961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:48.309498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 882/1714] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:48 2022] PUT /identity/v3/projects/c5679f30975246b19d849ff3aca53b1e/users/82ab2a3d306648259020716d3e7650ef/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:48.320346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:48.320346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:48.320707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:48.403685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] MFA Rules not processed for user `82ab2a3d306648259020716d3e7650ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:48.425714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:48.645525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:48.775587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:48.796738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 833/1715] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:48.963488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4088f36-4467-43b9-95cd-b03498137caf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:48.985778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:48.988480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:48.988986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:48.991745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:48.992750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:49.000748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:49.014717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:49.088487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:49.097465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 883/1716] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:48 2022] POST /identity/v3/projects => generated 381 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:49.118363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027e4c8-e3fc-4650-ada9-19eac5fe3780 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:49.371676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:49.372390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:49.372696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:49.372998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:49.373762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:49.385820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:49.463597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:49.474905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 834/1717] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:49 2022] POST /identity/v3/users => generated 297 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:49.486447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62d121-e61d-4599-8da6-018ff60e4c8c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:49.511081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:49.512108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:49.513525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:49.514872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:49.517434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:49.524731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:49.572577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:49.573092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 884/1718] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:49.591612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3dd64-cba2-4572-bf0a-68f88f334ca9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:49.619352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:49.620644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:49.621180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:49.621675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] PATH_INFO: `/v3/projects/4f3081e7a82b478790396813d4240d3b/users/700d2ae4d2ae416b8f26f24997c839bd/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:49.673221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3081e7a82b478790396813d4240d3b, user_id=700d2ae4d2ae416b8f26f24997c839bd, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:49.681632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:49.783137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 835/1719] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:49 2022] PUT /identity/v3/projects/4f3081e7a82b478790396813d4240d3b/users/700d2ae4d2ae416b8f26f24997c839bd/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:49.787100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:49.805563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99ceb0-8947-431e-a326-920552de5b0a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:50.012657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:50.014800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:50.016236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:50.019614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:50.021222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:50.025901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472465-4cbf-437f-bf55-8776843fc021 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:50.059763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 885/1720] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:50.060590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:50.075498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58743c-699e-4a48-a7a2-a04f74b4d5dd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:50.117734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:50.118744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:50.119334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:50.119334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] PATH_INFO: `/v3/projects/4f3081e7a82b478790396813d4240d3b/users/700d2ae4d2ae416b8f26f24997c839bd/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:50.137343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3081e7a82b478790396813d4240d3b, user_id=700d2ae4d2ae416b8f26f24997c839bd, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:50.155241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:50.226525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 836/1721] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:50 2022] PUT /identity/v3/projects/4f3081e7a82b478790396813d4240d3b/users/700d2ae4d2ae416b8f26f24997c839bd/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:50.239027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:50.239369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:50.239590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:50.317919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] MFA Rules not processed for user `700d2ae4d2ae416b8f26f24997c839bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:50.320116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:50.549432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 886/1722] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:50.561167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:50.564806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:50.834140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a24dcf-3e7a-480a-9a63-5db1d3a9c50f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:50.842350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:50.845678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:50.846714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:50.847179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:50.875209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:50.879871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181a5bd-6f1d-495c-9785-f5594e7079ac service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:50.911417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 837/1723] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:51.175792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:51.197820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8608e-bc3a-4e9b-96c1-f6a466360fc9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:51.218356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:51.233282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:51.233877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:51.234921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:51.268967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bd9c-cf02-461e-a049-2f16e3287dc1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:51.269857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bd9c-cf02-461e-a049-2f16e3287dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:51.270463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bd9c-cf02-461e-a049-2f16e3287dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:51.318378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a423bd9c-cf02-461e-a049-2f16e3287dc1 None None] MFA Rules not processed for user `b8d8d0c23f744a7b9ad311581f6ae39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:51.322420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a423bd9c-cf02-461e-a049-2f16e3287dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:51.541988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:51.552821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:51.557849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:51.566077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:51.617058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 838/1724] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:51.639037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 887/1725] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:51.641454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b804353-cd6b-4607-9386-421f1f9c7e23 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:51.641890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b804353-cd6b-4607-9386-421f1f9c7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:51.642320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b804353-cd6b-4607-9386-421f1f9c7e23 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:51.644121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 839/1726] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:51.650737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:51.654683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:51.756503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:51.756989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:51.757644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:51.798787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:51.802250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:51.877206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65837a-e621-4dfb-b1b4-3b2cc4baa2d4 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:51.893381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:51.894345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:51.894627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:51.894973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] PATH_INFO: `/v3/projects/fdad1c8493db416c8da8d75e35252313` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:51.898305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdad1c8493db416c8da8d75e35252313)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:51.912937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:52.060112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdad1c8493db416c8da8d75e35252313', 'request_id': 'req-79d301c7-057c-4f01-a555-a6c206f3523f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.078799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdad1c8493db416c8da8d75e35252313', 'request_id': 'req-79d301c7-057c-4f01-a555-a6c206f3523f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.104711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdad1c8493db416c8da8d75e35252313', 'request_id': 'req-79d301c7-057c-4f01-a555-a6c206f3523f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.129752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 840/1727] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:52.148712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] The token cache is being invalidate because project fdad1c8493db416c8da8d75e35252313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:52.149045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.161238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:52.162729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:52.179461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.200996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 888/1728] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:51 2022] DELETE /identity/v3/projects/fdad1c8493db416c8da8d75e35252313 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:52.207402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:52.213017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:52.236856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3c5dc-cde0-4d5f-b8f6-fc2278390b43 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:52.520788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:52.521831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:52.522048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:52.522359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] PATH_INFO: `/v3/users/aaaf6c2b064745648e96bcc3f968f627` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:52.530177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaaf6c2b064745648e96bcc3f968f627)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:52.532304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:52.551791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d301c7-057c-4f01-a555-a6c206f3523f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:52.579535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:52.588075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:52.590297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:52.591777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] PATH_INFO: `/v3/users/aa1e9c24db0c4ca48f38308a36d7b68b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:52.594995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1e9c24db0c4ca48f38308a36d7b68b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:52.599931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:52.694693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaaf6c2b064745648e96bcc3f968f627'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.721698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaaf6c2b064745648e96bcc3f968f627'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.729280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:52.749830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaaf6c2b064745648e96bcc3f968f627'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.755548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaaf6c2b064745648e96bcc3f968f627'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.772790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaaf6c2b064745648e96bcc3f968f627'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.802298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1e9c24db0c4ca48f38308a36d7b68b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.831606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1e9c24db0c4ca48f38308a36d7b68b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.870255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1e9c24db0c4ca48f38308a36d7b68b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.873554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1e9c24db0c4ca48f38308a36d7b68b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.873721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 841/1729] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:52 2022] DELETE /identity/v3/users/aaaf6c2b064745648e96bcc3f968f627 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:52.873721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:52.877584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7f4cbcdb-48d9-4f76-be7e-62dc97c2f54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1e9c24db0c4ca48f38308a36d7b68b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:52.881525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:52.905434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:52.964693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 889/1730] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:52 2022] DELETE /identity/v3/users/aa1e9c24db0c4ca48f38308a36d7b68b => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:52.973469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:52.973775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:52.974098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:53.036863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:53.049608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:53.209019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad62278-c6af-4cbd-817c-d7c9d8a37451 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:53.341762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 890/1731] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:53.449786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:53.452029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:53.497460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:53.498319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:53.498767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:53.499156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:53.502738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:53.545954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:53.550049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b47b0-7beb-4f9a-910a-efd99afe9755 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:53.590976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 842/1732] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:53.722830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c51eb-2168-4d80-af9d-8265bf8e29bf None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:53.736565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:53.740346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:53.740755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:53.741773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:53.807102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:53.809118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54109e-39e0-4fd8-8a0b-e1e1dda28ccd service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:53.833066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 891/1733] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:56.367691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:56.368313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:56.368561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:56.370475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 843/1734] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:04:56.401066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:56.401667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:56.401926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:56.471234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:56.473143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:56.499857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 892/1735] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:04:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:56.645725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:56.650917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:56.718829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035d913-8fe4-4f16-84ba-3bb1dbbf1cef None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:56.741994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:56.742780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:56.743054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:56.743316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] PATH_INFO: `/v3/projects/619bd4827d834a3996fbcdd43777dd10` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:56.748839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=619bd4827d834a3996fbcdd43777dd10)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:56.750948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:56.862357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:56.864259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:56.886980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1b1bc-35e3-4815-8b76-ef1a19db9f25 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:56.905454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:56.906381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:56.906902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:56.907326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:56.926590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:56.930434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:56.940082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619bd4827d834a3996fbcdd43777dd10', 'request_id': 'req-edd694f7-3515-4eee-96a3-c3d895224be8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:56.956675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619bd4827d834a3996fbcdd43777dd10', 'request_id': 'req-edd694f7-3515-4eee-96a3-c3d895224be8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:56.961597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 893/1736] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:04:56.963694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619bd4827d834a3996fbcdd43777dd10', 'request_id': 'req-edd694f7-3515-4eee-96a3-c3d895224be8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:56.988116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] The token cache is being invalidate because project 619bd4827d834a3996fbcdd43777dd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:56.988572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:56.998817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.000682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 844/1737] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:56 2022] DELETE /identity/v3/projects/619bd4827d834a3996fbcdd43777dd10 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:57.004950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:57.022833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15d8bd-b7c5-4cf1-ae2e-73d7baa806b2 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:57.145282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:57.222386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:57.223752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.224243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.225134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] PATH_INFO: `/v3/users/9905c4cf65f244a8b289d1aaf0340fd1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.227208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9905c4cf65f244a8b289d1aaf0340fd1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:57.233213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:57.377973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9905c4cf65f244a8b289d1aaf0340fd1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.386010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:57.389701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:57.403764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9905c4cf65f244a8b289d1aaf0340fd1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.409448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd694f7-3515-4eee-96a3-c3d895224be8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:57.413383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9905c4cf65f244a8b289d1aaf0340fd1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.414515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9905c4cf65f244a8b289d1aaf0340fd1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.415970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3a07169f-1a9f-4746-9359-a601b18a2f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9905c4cf65f244a8b289d1aaf0340fd1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.448486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:57.450106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.450685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.451409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] PATH_INFO: `/v3/projects/4f6581f99b314a25a57c25ce087b60f9` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.452230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 894/1738] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:57 2022] DELETE /identity/v3/users/9905c4cf65f244a8b289d1aaf0340fd1 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:57.455964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6581f99b314a25a57c25ce087b60f9)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:57.459065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:57.467111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:57.541766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6581f99b314a25a57c25ce087b60f9', 'request_id': 'req-75243f3b-09cb-4002-9452-51452871f048'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.552536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6581f99b314a25a57c25ce087b60f9', 'request_id': 'req-75243f3b-09cb-4002-9452-51452871f048'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.561039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6581f99b314a25a57c25ce087b60f9', 'request_id': 'req-75243f3b-09cb-4002-9452-51452871f048'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.583148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] The token cache is being invalidate because project 4f6581f99b314a25a57c25ce087b60f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:57.583550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.587726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-75243f3b-09cb-4002-9452-51452871f048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:57.591198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 845/1739] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:57 2022] DELETE /identity/v3/projects/4f6581f99b314a25a57c25ce087b60f9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:57.596692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.597420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.597770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.629540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:57.631205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:57.705934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 895/1740] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:57.714881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.715179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.715459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.751235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:57.752805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:57.762827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 846/1741] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:57.771601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:57.799802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1c53-50c9-491c-b372-2c7132be2065 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:57.805856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:57.806802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.807237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.807658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.809070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:57.809526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:57.811310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:57.834123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:57.834361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 896/1742] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:04:57.844745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ff114-9764-45ac-9f6c-09b39229cc5a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:57.858250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:57.859263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.859736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.860208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.861168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:57.863564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:57.896355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 847/1743] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:57 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:57.897796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:57.907965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181507d7-4bf9-41fc-90ac-4b9813f9f53b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:57.938917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:57.972687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:57.973552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:57.973840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:57.974116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:57.974820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:57.977895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.007153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 897/1744] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:57 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:58.008061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.020338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206aad9-af39-4afd-8939-edaafa936386 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.035553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.036652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.037189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.037675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.038564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.040256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.041158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.043624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.061099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdd4e-e950-4d6f-abea-43e5e6fba226 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.065129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 848/1745] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:58.066311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.067403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.068309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.068596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.068876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] PATH_INFO: `/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.070411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00dd9fb71448441b9189a0f2b6d62eaf)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.072547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.082781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fffa9-8809-4a08-a034-4d82cd1a76b7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.089448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.090135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.090420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.090838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.091831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.094269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.110799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 849/1746] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:58.114696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.125819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b99ab-c205-4d33-9ed2-2d615df29e57 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.136669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00dd9fb71448441b9189a0f2b6d62eaf', 'request_id': 'req-49243198-0770-49b7-85bf-e10418c56785'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:58.143427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00dd9fb71448441b9189a0f2b6d62eaf', 'request_id': 'req-49243198-0770-49b7-85bf-e10418c56785'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:58.149297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00dd9fb71448441b9189a0f2b6d62eaf', 'request_id': 'req-49243198-0770-49b7-85bf-e10418c56785'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:58.164351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] The token cache is being invalidate because project 00dd9fb71448441b9189a0f2b6d62eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:04:58.164641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:58.165394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-49243198-0770-49b7-85bf-e10418c56785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:58.167394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 898/1747] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:04:58 2022] DELETE /identity/v3/projects/00dd9fb71448441b9189a0f2b6d62eaf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:58.173256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d88e-9704-472b-addb-062dac5af88e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.173706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d88e-9704-472b-addb-062dac5af88e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.174131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12d88e-9704-472b-addb-062dac5af88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.191638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.192645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.193028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.193393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] PATH_INFO: `/v3/projects/88faf090e8944ba89a087192e8ee60f1/users/0210218b331d46de8bcae52929c32b0a/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.195001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3d12d88e-9704-472b-addb-062dac5af88e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:58.196280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3d12d88e-9704-472b-addb-062dac5af88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.204871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 899/1748] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:58.210656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] RBAC: Authorizing `identity:create_grant(project_id=88faf090e8944ba89a087192e8ee60f1, user_id=0210218b331d46de8bcae52929c32b0a, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.213528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf591e5-0ef3-46f0-b526-01c52f3291be admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.214205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.214501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.214779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.231834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 850/1749] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:58 2022] PUT /identity/v3/projects/88faf090e8944ba89a087192e8ee60f1/users/0210218b331d46de8bcae52929c32b0a/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:58.235058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:58.236283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.243101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.243716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.243996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.275263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] MFA Rules not processed for user `0210218b331d46de8bcae52929c32b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:58.276485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.289493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:58.315844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 900/1750] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:58.325051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.335311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac93c37-84d3-40b6-a874-b847dc079042 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.340471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.341257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.341723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.342063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.343365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:04:58.343850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.345549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.361914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 901/1751] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:04:58.363125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.369656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 851/1752] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:58.373898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.374572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1ac5-ed3e-4053-9069-5c7198b8575c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.375294 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.379439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.380278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.380580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.380942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.381856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.384591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.388612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514f3dc-919a-45fd-bf8f-e75b3a756757 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.399243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.400436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.400719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.401023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.401733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.403704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.411921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 902/1753] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:58.416122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.421754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:58.431493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 852/1754] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:58.435594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.483884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a16ca5-4e86-4127-b327-8a18804a3afb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.500256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53c567-d481-4128-a2d4-b4c0c1b4a7ff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.506184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.507045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.507365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.507681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.509072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.511067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.532241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 853/1755] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:04:58.536132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.544307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.545059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.545346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.545640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.546333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.547303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd744cc-4b0c-441c-952d-93e3691a24df admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.548267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.552704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.553483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.553746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.554054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.554759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.557002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.571691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 854/1756] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:04:58.571884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 903/1757] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:58.572971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.573917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.582600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6bb83-fa28-4d60-8b28-196449eb169a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.583724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e2023-1905-4993-8f45-ecebea9e4afc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.587244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.588129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.588374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.588483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.588668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] PATH_INFO: `/v3/projects/d7573362ba654efd9e6e8e083fbcdb78/users/8932cd13715841c9893db179afc4ed90/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.589191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.589472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.589748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.590478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.592096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.606137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 904/1758] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:04:58.606326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7573362ba654efd9e6e8e083fbcdb78, user_id=8932cd13715841c9893db179afc4ed90, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.607150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.607941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.617365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68926e-0a4d-4e3e-a392-d512761aaefa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.622791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 855/1759] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:58 2022] PUT /identity/v3/projects/d7573362ba654efd9e6e8e083fbcdb78/users/8932cd13715841c9893db179afc4ed90/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:58.626488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.672989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.673671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.673972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.674123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:58.674268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.674952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.676568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.686916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 905/1760] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:58.690253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6993d-5f35-4e9d-873f-1fc71bc8bae8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.691086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.696008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.696728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.697047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.697324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.698234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.699904 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.700839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2fb3-30a9-4c90-a124-5d4aec442c13 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.706056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.706946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.707268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.707516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] PATH_INFO: `/v3/projects/0f617fa5c70d45d490098107db9d7d57/users/9274d9a18b0f405fb090f93b9ab9f063/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.710825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 856/1761] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:58.714894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.724448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f617fa5c70d45d490098107db9d7d57, user_id=9274d9a18b0f405fb090f93b9ab9f063, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.724870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a7b27-ab98-42c9-ba27-294e6dbf7f0a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.727213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd12f1b-3be6-4d2e-a8ab-1c0b35f5bff4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.729894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.730773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.731056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.731332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] PATH_INFO: `/v3/projects/d7573362ba654efd9e6e8e083fbcdb78/users/8932cd13715841c9893db179afc4ed90/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.733367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7573362ba654efd9e6e8e083fbcdb78, user_id=8932cd13715841c9893db179afc4ed90, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.736060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38347264-0bc2-49d0-9a8a-fa163e591e3f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.742132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 906/1762] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:58 2022] PUT /identity/v3/projects/0f617fa5c70d45d490098107db9d7d57/users/9274d9a18b0f405fb090f93b9ab9f063/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:58.749483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 857/1763] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:58 2022] PUT /identity/v3/projects/d7573362ba654efd9e6e8e083fbcdb78/users/8932cd13715841c9893db179afc4ed90/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:04:58.753447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.753709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.753993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.760100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:58.760569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.760870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.761177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.787474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] MFA Rules not processed for user `9274d9a18b0f405fb090f93b9ab9f063`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:58.788784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.802141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] MFA Rules not processed for user `8932cd13715841c9893db179afc4ed90`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:58.803500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.859711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 907/1764] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:58.865443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.891244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 858/1765] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:04:58.899743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.901705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:58.941414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893fb1e-43ec-4592-acec-8ea736e5e84e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.946319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.946999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.947280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.947556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.948257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.950252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.967212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0f01-ed37-4489-a52c-cde2474e8ccf None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:58.971401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:58.972117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:58.972632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:58.972924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:58.976274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 908/1766] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:58.980189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:58.983621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:58.985140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:58.995301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 859/1767] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:04:59.047093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050139-ec3d-4085-b3c5-ba31819972ae admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.052932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.054083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.054385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.054818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.056120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.058380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.083642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 909/1768] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:04:58 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:04:59.087814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.110983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ca3-d1be-49ea-8d8e-33a2cafe7a87 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.120324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.122233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.122483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.122764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.123838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.129005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.141593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 860/1769] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:59.146432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.159410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed063bf-06a4-4c2e-8f11-09a45bacf8bc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.167070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.168470 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.168661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.168968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] PATH_INFO: `/v3/projects/0a5c311f510f4856a11963cefe599508/users/f72c244361134536b869d600c16cb686/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.172305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:59.199116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5c311f510f4856a11963cefe599508, user_id=f72c244361134536b869d600c16cb686, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.202088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.222424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 910/1770] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:59 2022] PUT /identity/v3/projects/0a5c311f510f4856a11963cefe599508/users/f72c244361134536b869d600c16cb686/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:59.227578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.239551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e26678-9d3e-4c8a-b388-fc3dd266ee8d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.296080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:59.312250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.313095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.313490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.313857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.314795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.316489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ba0f9-e051-4a70-b2ec-b99cefe770d7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.328954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 861/1771] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:04:59.332788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.345638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac3acb-5a0b-47dc-8dbd-8d80bdf03657 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.355619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.356625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.356922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.357334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] PATH_INFO: `/v3/projects/0a5c311f510f4856a11963cefe599508/users/f72c244361134536b869d600c16cb686/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.359312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5c311f510f4856a11963cefe599508, user_id=f72c244361134536b869d600c16cb686, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.361964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.378939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 911/1772] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:04:59 2022] PUT /identity/v3/projects/0a5c311f510f4856a11963cefe599508/users/f72c244361134536b869d600c16cb686/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:59.389871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.390460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.390578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.458826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] MFA Rules not processed for user `f72c244361134536b869d600c16cb686`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:59.461498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:59.495930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.583953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 862/1773] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:59.589578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364f1c0-96e4-4819-b026-ec1f3f807579 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.594262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.597662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.599013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.599790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.600077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] PATH_INFO: `/v3/users/0210218b331d46de8bcae52929c32b0a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.602077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0210218b331d46de8bcae52929c32b0a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.604935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.660019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6168bb-5268-441d-8c0c-c466265db950 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.663860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.664545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.664831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.665143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0210218b331d46de8bcae52929c32b0a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:59.665253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.673686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0210218b331d46de8bcae52929c32b0a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:59.674366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.675852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fe08-20af-4ca9-85a8-cf5e29b9abb5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.681553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0210218b331d46de8bcae52929c32b0a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:59.682215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0210218b331d46de8bcae52929c32b0a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:59.682837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0210218b331d46de8bcae52929c32b0a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:04:59.685876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 863/1774] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:04:59.701734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 912/1775] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:04:59 2022] DELETE /identity/v3/users/0210218b331d46de8bcae52929c32b0a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:04:59.706958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.707252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.707529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.731786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:04:59.733045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:59.794054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 864/1776] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:04:59.805940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:04:59.807555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:04:59.861589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:04:59.866093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe03481-7444-4823-840d-32c0a9bc8193 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:04:59.870785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:04:59.873072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:04:59.873072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:04:59.873072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:04:59.883416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:04:59.884896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:04:59.895471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 913/1777] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:00.241311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:00.242853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:00.258062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1adbf0-0c0f-4d7b-b242-a57911f8e612 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:00.266995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:00.268816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:00.269210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:00.269554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] PATH_INFO: `/v3/projects/88faf090e8944ba89a087192e8ee60f1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:00.271140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88faf090e8944ba89a087192e8ee60f1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:00.273258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:00.283536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:00.341228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:00.343129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:00.344094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88faf090e8944ba89a087192e8ee60f1', 'request_id': 'req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.351183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88faf090e8944ba89a087192e8ee60f1', 'request_id': 'req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.356360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac0b86-27dc-41af-b310-d6c6439b302a service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:00.358540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88faf090e8944ba89a087192e8ee60f1', 'request_id': 'req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.374730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] The token cache is being invalidate because project 88faf090e8944ba89a087192e8ee60f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:00.375305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.375966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.377784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 865/1778] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:00 2022] DELETE /identity/v3/projects/88faf090e8944ba89a087192e8ee60f1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:00.382172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:00.383515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:00.426703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:00.428058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:00.428552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:00.429153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] PATH_INFO: `/v3/users/9274d9a18b0f405fb090f93b9ab9f063` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:00.430957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9274d9a18b0f405fb090f93b9ab9f063)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:00.434604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:00.458824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7ac45-02f6-4c41-a9a1-5ab057de6f78 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:00.465706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:00.466633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:00.466985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:00.467254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] PATH_INFO: `/v3/users/8932cd13715841c9893db179afc4ed90` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:00.468970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8932cd13715841c9893db179afc4ed90)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:00.470959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:00.490392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9274d9a18b0f405fb090f93b9ab9f063'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.500328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9274d9a18b0f405fb090f93b9ab9f063'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.510398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9274d9a18b0f405fb090f93b9ab9f063'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.511265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9274d9a18b0f405fb090f93b9ab9f063'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.512011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8be7e033-70f8-40f1-9b19-b0f82f460787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9274d9a18b0f405fb090f93b9ab9f063'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.527392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8932cd13715841c9893db179afc4ed90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.531579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 914/1779] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:00 2022] DELETE /identity/v3/users/9274d9a18b0f405fb090f93b9ab9f063 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:00.536003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:00.536371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:00.536722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:00.537674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8932cd13715841c9893db179afc4ed90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.546721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8932cd13715841c9893db179afc4ed90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.547376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8932cd13715841c9893db179afc4ed90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.547963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8932cd13715841c9893db179afc4ed90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:00.558343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:00.559584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:00.567284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 866/1780] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:00 2022] DELETE /identity/v3/users/8932cd13715841c9893db179afc4ed90 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:00.637300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 915/1781] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:00.647185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:00.649573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:00.674729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:00.722542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea20e7-49c2-4d59-844f-fe50ee38d4c7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:00.726445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:00.727317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:00.727473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:00.727748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:00.829663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:00.831647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945edf35-3b0b-4a2f-83f7-b3c7630cf5d9 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:00.845834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 867/1782] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:01.009492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.011160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.024218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022f4fc-7366-41cd-a473-42d1efb7d381 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.028863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.029828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.030118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.030419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] PATH_INFO: `/v3/projects/d7573362ba654efd9e6e8e083fbcdb78` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.032212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7573362ba654efd9e6e8e083fbcdb78)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.034021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.043131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:01.105418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7573362ba654efd9e6e8e083fbcdb78', 'request_id': 'req-6ffdd876-5dd9-449d-8970-47a9033f6e1b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.111930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7573362ba654efd9e6e8e083fbcdb78', 'request_id': 'req-6ffdd876-5dd9-449d-8970-47a9033f6e1b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.117949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7573362ba654efd9e6e8e083fbcdb78', 'request_id': 'req-6ffdd876-5dd9-449d-8970-47a9033f6e1b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.132743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] The token cache is being invalidate because project d7573362ba654efd9e6e8e083fbcdb78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:01.132862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.133560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.136939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 916/1783] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:01 2022] DELETE /identity/v3/projects/d7573362ba654efd9e6e8e083fbcdb78 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:01.140379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825d4c-6893-4e60-8482-317d240b5df9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.140676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825d4c-6893-4e60-8482-317d240b5df9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.141396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825d4c-6893-4e60-8482-317d240b5df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.159267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3f825d4c-6893-4e60-8482-317d240b5df9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:01.160925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3f825d4c-6893-4e60-8482-317d240b5df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.251382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 868/1784] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:01.251488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.251488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.261590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c94b25-1059-47f2-a482-48c067251850 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.261889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c94b25-1059-47f2-a482-48c067251850 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.263082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c94b25-1059-47f2-a482-48c067251850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.283381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdd876-5dd9-449d-8970-47a9033f6e1b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.288939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.289469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-00c94b25-1059-47f2-a482-48c067251850 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:01.289890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.290122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.290402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] PATH_INFO: `/v3/projects/0f617fa5c70d45d490098107db9d7d57` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.290688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-00c94b25-1059-47f2-a482-48c067251850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.291809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f617fa5c70d45d490098107db9d7d57)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.294554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.306445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 869/1785] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:01.309223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c94b25-1059-47f2-a482-48c067251850 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.322252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c94b25-1059-47f2-a482-48c067251850 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.331914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.332982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.333270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.333554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.334580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:01.334960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.337063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.355240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 870/1786] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:01.356825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.371062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2131-0c5f-40d7-bb4f-8023110924ba admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.378060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f617fa5c70d45d490098107db9d7d57', 'request_id': 'req-5a67bdbe-3057-49df-a833-9693ce8c2a68'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.390019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f617fa5c70d45d490098107db9d7d57', 'request_id': 'req-5a67bdbe-3057-49df-a833-9693ce8c2a68'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.406950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f617fa5c70d45d490098107db9d7d57', 'request_id': 'req-5a67bdbe-3057-49df-a833-9693ce8c2a68'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.416893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] The token cache is being invalidate because project 0f617fa5c70d45d490098107db9d7d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:01.417214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.418628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.420575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 917/1787] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:01 2022] DELETE /identity/v3/projects/0f617fa5c70d45d490098107db9d7d57 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:01.425863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.427627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.430289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:01.450068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.450969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.451350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.451735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.452679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.454743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.486361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 871/1788] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:01.493105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.496787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.522178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67bdbe-3057-49df-a833-9693ce8c2a68 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.551789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:01.590247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283888ad-d072-4c08-8918-00fbf04ce7dc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.597006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.598200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.598582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.598945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.599585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.599840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.600879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.601108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.602403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.608101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] PATH_INFO: `/v3/users/f72c244361134536b869d600c16cb686` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.609987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72c244361134536b869d600c16cb686)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.613173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.629420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 872/1789] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:01.633252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.643969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03367379-8112-46b1-9f1b-857f89cf31bd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.657789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.658441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.658718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.659040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.659724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.661396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.676033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 873/1790] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:01.676408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72c244361134536b869d600c16cb686'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.678601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.690927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72c244361134536b869d600c16cb686'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.695245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b007-796c-4c07-af1b-c1c70391332a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.700729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.701569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.702188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.702462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.703199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.704819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.704968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72c244361134536b869d600c16cb686'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.706371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72c244361134536b869d600c16cb686'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.708363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7cdfd3c6-e88e-49d9-aff3-1fcaea95abbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72c244361134536b869d600c16cb686'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:01.721229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 874/1791] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:01.722855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:01.724610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.738890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 918/1792] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:01 2022] DELETE /identity/v3/users/f72c244361134536b869d600c16cb686 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:01.764189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda11ff0-963a-40f2-8d71-557f61551024 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:01.821561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:01.822407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.822744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.823023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] PATH_INFO: `/v3/projects/7993a0e6bb114808a5bf848f42696b3a/users/d1fdd2b3921344a2917d41c96a65397c/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.842912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7993a0e6bb114808a5bf848f42696b3a, user_id=d1fdd2b3921344a2917d41c96a65397c, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:01.846861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:01.864083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 875/1793] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:01 2022] PUT /identity/v3/projects/7993a0e6bb114808a5bf848f42696b3a/users/d1fdd2b3921344a2917d41c96a65397c/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:01.875172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:01.875820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:01.876225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:01.910128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] MFA Rules not processed for user `d1fdd2b3921344a2917d41c96a65397c`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:01.911896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:01.984858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 919/1794] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:01.989798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.013427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.045833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.047489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.077791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724fdf5-e506-4fd3-a88f-f8e876b075ac admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.082439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.083116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.083401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.083677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.084385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.086442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.097910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.128001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 876/1795] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:01 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:02.129793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.133626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b14881-9f33-4bcd-86b0-2c649751115d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.189427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.190309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.190590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.190870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] PATH_INFO: `/v3/projects/0a5c311f510f4856a11963cefe599508` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.192065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5c311f510f4856a11963cefe599508)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.193319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907d28-5d44-4fd9-b0b7-15676c5a3d91 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.194426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.198458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.199264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.199548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.199952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.200644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.202630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.228015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 877/1796] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:02.229127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.239090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbaf31-4767-4cb9-9ab8-15ba7517f299 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.246685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5c311f510f4856a11963cefe599508', 'request_id': 'req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:02.253228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5c311f510f4856a11963cefe599508', 'request_id': 'req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:02.259110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5c311f510f4856a11963cefe599508', 'request_id': 'req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:02.272665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] The token cache is being invalidate because project 0a5c311f510f4856a11963cefe599508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:02.272821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:02.273547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7a4ee3cf-4afa-49ed-a2a4-d139053be17a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:02.275156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 920/1797] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:02 2022] DELETE /identity/v3/projects/0a5c311f510f4856a11963cefe599508 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:02.280154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37afc-48a6-4892-bcce-b0bbb4e06939 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.280448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37afc-48a6-4892-bcce-b0bbb4e06939 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.280818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37afc-48a6-4892-bcce-b0bbb4e06939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.296550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.297267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.297554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.297830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.298553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7ac37afc-48a6-4892-bcce-b0bbb4e06939 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:02.298654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.299768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7ac37afc-48a6-4892-bcce-b0bbb4e06939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.300192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.314097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 878/1798] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:02.314267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 921/1799] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:02.318180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.320723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.321503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.321792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.322066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.336186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb637-8b8e-44f1-af1d-7b6772198a5e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.341159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:02.341449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.342729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.342885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.343013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.343288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] PATH_INFO: `/v3/projects/5935391fbf764a61b70e903da432848b/users/50185bbc78a5478c83a0ff43a50757b5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.351974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 922/1800] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:02.356434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.359389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5935391fbf764a61b70e903da432848b, user_id=50185bbc78a5478c83a0ff43a50757b5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.362151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.367072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6fb7-d48d-4a18-9b51-2c7768aa629e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.372537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.373575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.373893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.374172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.374903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:02.375216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.376834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.377164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 879/1801] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:02 2022] PUT /identity/v3/projects/5935391fbf764a61b70e903da432848b/users/50185bbc78a5478c83a0ff43a50757b5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:02.381193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.390000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.390462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 923/1802] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:02.395183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.405527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9656f8-10b7-4310-a2bd-37882c8e40b6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.451538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.462495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d537e45-33a8-4443-83da-d144fe7c9037 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.468376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.469117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.469404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.469678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.470410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.472225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.472716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.473503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.474028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.474395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.475665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.477698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.486416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 880/1803] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:02.488027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.500542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85210-1734-4c02-9c00-1709461c91de admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.505742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.506991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.507101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.507338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] PATH_INFO: `/v3/projects/5935391fbf764a61b70e903da432848b/users/50185bbc78a5478c83a0ff43a50757b5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.509433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5935391fbf764a61b70e903da432848b, user_id=50185bbc78a5478c83a0ff43a50757b5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.510445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 924/1804] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:02.512083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9eb75-4a50-493b-a163-3a512a4f5ef0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.514485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.523358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 881/1805] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:02 2022] PUT /identity/v3/projects/5935391fbf764a61b70e903da432848b/users/50185bbc78a5478c83a0ff43a50757b5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:02.536125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.536562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.537298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.566271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] MFA Rules not processed for user `50185bbc78a5478c83a0ff43a50757b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:02.567528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.585294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845494b-1e7d-454e-8b58-63281bed9706 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.638441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 882/1806] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:02.651152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.652553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.655667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.656460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.656808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.657186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.657953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.659967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.684822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 925/1807] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:02.685466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.695348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b792-891c-4aec-b384-4cc609a088cd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.700244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.700954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.701240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.701520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.702248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.703868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.715260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 926/1808] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:02.719417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.721440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d468012-61d5-4fcd-9e67-c63644f29128 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.725858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.726524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.726818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.727089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.729727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b88bc-bd44-42fd-9a32-cac1707fe7e4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.734595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.735354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.735709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.736054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.736952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.738748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.739313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.740774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.751721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 883/1809] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:05:02.753779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 927/1810] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:02.761873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.776673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9d25-8ae0-405c-8e95-81fd2170739e service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.791903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.793238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.793238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.793442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] PATH_INFO: `/v3/projects/04a0cdc4cfe245a195d240f256eccc60/users/99178658990d46588215c9f0352509c0/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.826859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a0cdc4cfe245a195d240f256eccc60, user_id=99178658990d46588215c9f0352509c0, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.829673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a20087-4fd5-4b95-8ba3-315cc5c0be86 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:02.833352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.856474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 884/1811] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:02 2022] PUT /identity/v3/projects/04a0cdc4cfe245a195d240f256eccc60/users/99178658990d46588215c9f0352509c0/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:02.857417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:02.866614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.866910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.867202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.868775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75576e6-2168-4c61-9010-11feeba1157d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:02.906821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] MFA Rules not processed for user `99178658990d46588215c9f0352509c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:02.908097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.918606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:02.931047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:02.931742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:02.932019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:02.932291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:02.937936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:02.988896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 885/1812] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:02.998348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:02.999847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:03.004798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:03.007067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:03.010643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 928/1813] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:05:03.019803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e90a-f30e-40b4-a0bd-2090fc1c0cb3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:03.025192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:03.026304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.026304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.026523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.035924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:03.037416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74881572-ef11-4b97-80d3-57b7d98cef80 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:03.047166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 886/1814] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:03.068585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:03.146384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feed6a2-53bc-4536-becd-c6cc6b05f14c service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:03.151283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:03.152330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.152658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.152923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] PATH_INFO: `/v3/users/99178658990d46588215c9f0352509c0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.154142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99178658990d46588215c9f0352509c0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:03.156282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:03.211981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99178658990d46588215c9f0352509c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.221395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99178658990d46588215c9f0352509c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.229410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99178658990d46588215c9f0352509c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.230087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99178658990d46588215c9f0352509c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.230695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99178658990d46588215c9f0352509c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.248685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 929/1815] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:03 2022] DELETE /identity/v3/users/99178658990d46588215c9f0352509c0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:03.253271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.253559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.253831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.271161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:03.272525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:03.343717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 887/1816] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:03.355908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:03.358367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:03.382580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:03.422597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab5273-8159-4a7c-9a93-09770f8e3c18 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:03.426530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:03.427190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.427474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.427751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.436851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:03.438327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b51629-2fb4-4c2f-b55b-6d6ed9715973 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:03.448688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 930/1817] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:03.740831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:03.742496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:03.754852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b8136-d9b1-4ae4-a4f2-f5c0e7eaaada None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:03.768511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:03.770780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.770780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.770780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] PATH_INFO: `/v3/projects/04a0cdc4cfe245a195d240f256eccc60` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.774721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a0cdc4cfe245a195d240f256eccc60)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:03.776692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:03.826974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a0cdc4cfe245a195d240f256eccc60', 'request_id': 'req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.833313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a0cdc4cfe245a195d240f256eccc60', 'request_id': 'req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.840268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a0cdc4cfe245a195d240f256eccc60', 'request_id': 'req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.843429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:03.856043 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] The token cache is being invalidate because project 04a0cdc4cfe245a195d240f256eccc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:03.856332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.856993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e64f3b9-c2c2-4ffa-9059-08d923911e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:03.858697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 888/1818] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:03 2022] DELETE /identity/v3/projects/04a0cdc4cfe245a195d240f256eccc60 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:03.864416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.864954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.865156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.884215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:03.885483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:03.963973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 931/1819] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:03.972498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:03.972789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:03.973097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:03.993044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:03.994310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:04.002523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 889/1820] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:04.007141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.017463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e689-538a-4c82-8a50-893a16cba28b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.022138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.022798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.023079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.023355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.024096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:04.024404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.026056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.037840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 932/1821] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:04.042596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.052210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdac2a-924d-485c-b7f6-7eb90dad05df None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.057283 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.057956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.058239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.058540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.059282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.061322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.088768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 890/1822] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:04 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:04.095296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.108033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f819ee-cf6a-4ca1-bc18-3e18b79dbd21 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.169521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.170116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.170396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.170671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.171359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.173330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.194681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 933/1823] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:04 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:04.198778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.210621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6c8ff-f22e-4d0b-86c6-a57c1fb871c2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.216148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.216823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.217135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.217800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.218505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.220147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.231149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 891/1824] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:04.235346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.245504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b908c-9748-4891-953a-54587b4a87ca admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.250885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.251549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.251826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.252102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.252831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.254494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db636b-89a7-4075-8b14-c13c8612c29f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.267353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 934/1825] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:04.271511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.282378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1428fc8-ed52-4c59-a5eb-5757948d80a0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.288421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.289345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.289624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.289900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] PATH_INFO: `/v3/projects/fa2c71e4d23a4b54996f08230388a87b/users/f3fb076c343c42e889bea217042a78bd/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.305819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2c71e4d23a4b54996f08230388a87b, user_id=f3fb076c343c42e889bea217042a78bd, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.308586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.323553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 892/1826] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:04 2022] PUT /identity/v3/projects/fa2c71e4d23a4b54996f08230388a87b/users/f3fb076c343c42e889bea217042a78bd/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:04.335184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.335565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.335794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.366456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] MFA Rules not processed for user `f3fb076c343c42e889bea217042a78bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:04.367371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:04.380084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:04.436877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 935/1827] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:04.448597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:04.450615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:04.507335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104f5aa-baf0-49f8-9565-8b9ca1afba36 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:04.511031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:04.511683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:04.511963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:04.512236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:04.520633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:04.522125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957a821-cb8d-41ba-b033-3f308067c105 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:04.531162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 893/1828] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:06.880024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:06.881626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:06.948763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84723f1b-b899-4c31-aebb-55669f64156a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:06.954062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:06.954991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:06.955269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:06.955539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] PATH_INFO: `/v3/users/d1fdd2b3921344a2917d41c96a65397c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:06.956860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1fdd2b3921344a2917d41c96a65397c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:06.958814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:07.016700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1fdd2b3921344a2917d41c96a65397c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.032556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1fdd2b3921344a2917d41c96a65397c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.044887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1fdd2b3921344a2917d41c96a65397c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.045726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1fdd2b3921344a2917d41c96a65397c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.046462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1fdd2b3921344a2917d41c96a65397c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.071909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 936/1829] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:06 2022] DELETE /identity/v3/users/d1fdd2b3921344a2917d41c96a65397c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:07.076601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:07.077120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:07.077320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:07.095787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:07.097113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:07.160739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 894/1830] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:07.170650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:07.172073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:07.213430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:07.233649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfcfcf-8f97-4b6e-83cc-ddd6e0e9881d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:07.237853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:07.238516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:07.238804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:07.239844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:07.249945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:07.251515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:07.264352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 937/1831] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:07.613678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:07.616400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:07.633197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d94c00-9d2c-49dd-9001-d042342aad2d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:07.639127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:07.640016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:07.640269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:07.640549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] PATH_INFO: `/v3/projects/7993a0e6bb114808a5bf848f42696b3a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:07.641848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7993a0e6bb114808a5bf848f42696b3a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:07.643779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:07.676962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:07.693191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7993a0e6bb114808a5bf848f42696b3a', 'request_id': 'req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.699693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7993a0e6bb114808a5bf848f42696b3a', 'request_id': 'req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.705841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7993a0e6bb114808a5bf848f42696b3a', 'request_id': 'req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.719809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] The token cache is being invalidate because project 7993a0e6bb114808a5bf848f42696b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:07.720092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.720744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.722402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 895/1832] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:07 2022] DELETE /identity/v3/projects/7993a0e6bb114808a5bf848f42696b3a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:07.727230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:07.737353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920232f-906c-4942-8c5f-e26e9c22f4ce service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:07.817189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:07.818077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:07.818353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:07.818623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] PATH_INFO: `/v3/users/50185bbc78a5478c83a0ff43a50757b5` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:07.820335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] RBAC: Authorizing `identity:delete_user(user_id=50185bbc78a5478c83a0ff43a50757b5)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:07.822282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:07.872341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50185bbc78a5478c83a0ff43a50757b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.880588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50185bbc78a5478c83a0ff43a50757b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.888576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50185bbc78a5478c83a0ff43a50757b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.889288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50185bbc78a5478c83a0ff43a50757b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.889882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3b43d526-0123-4d7a-adbd-29456bc3083e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50185bbc78a5478c83a0ff43a50757b5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:07.908859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 938/1833] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:07 2022] DELETE /identity/v3/users/50185bbc78a5478c83a0ff43a50757b5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:08.250590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.252102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:08.266150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca24f-ac1c-4a02-a3e1-5c50aa5c1981 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.325161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.326024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.326301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.326570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] PATH_INFO: `/v3/projects/5935391fbf764a61b70e903da432848b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.327754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5935391fbf764a61b70e903da432848b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.329726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.381833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5935391fbf764a61b70e903da432848b', 'request_id': 'req-217e338f-dbb5-455d-b411-a5d76873f251'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:08.388532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5935391fbf764a61b70e903da432848b', 'request_id': 'req-217e338f-dbb5-455d-b411-a5d76873f251'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:08.395102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5935391fbf764a61b70e903da432848b', 'request_id': 'req-217e338f-dbb5-455d-b411-a5d76873f251'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:08.415397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] The token cache is being invalidate because project 5935391fbf764a61b70e903da432848b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:08.415771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:08.416459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-217e338f-dbb5-455d-b411-a5d76873f251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:08.418448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 896/1834] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:08 2022] DELETE /identity/v3/projects/5935391fbf764a61b70e903da432848b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:08.423826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.424183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.424523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.442966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:08.444376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:08.498237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:08.515649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 939/1835] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:08.519262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.519546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.519818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.538918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:08.540116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:08.547970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 897/1836] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:08.552203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.562246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a747f86-a136-4162-aab3-34841414b2d2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.566912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.567582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.567863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.568138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.569038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:08.569386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.570993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.583840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 940/1837] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:08.587930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.597678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8f0f-a741-490a-aad1-36f1729b5bcf None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.602957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.603617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.603895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.604162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.604879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.606816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.636047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 898/1838] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:08 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:08.640339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.655200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb3201-a41e-4186-ac20-5a91b773dc9e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.724831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.726536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.726655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.726874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.727771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.729860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.755856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 941/1839] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:08 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:08.760169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.771354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35073295-1e6a-45f8-b7df-3b1f5498ddae admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.776692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.777540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.777956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.778344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.779180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.781001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.792666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 899/1840] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:08.796773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.807850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f91b0-dbb4-4d18-96bc-29cfade4f604 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.813708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.814502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.814902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.815278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.816423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.819108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4d0f-e98f-4579-91be-db650c86ff20 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.830290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 942/1841] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:08.835156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:08.845507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6d5dd-124e-4ff3-ae8d-0a2779e795e4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:08.850360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:08.851251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.851525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.851784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] PATH_INFO: `/v3/projects/4b7d6516520840eea06fd2cb8888da7e/users/97857857e4524cfdbac3dfec7f953059/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.854513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:08.871850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7d6516520840eea06fd2cb8888da7e, user_id=97857857e4524cfdbac3dfec7f953059, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:08.874592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:08.889926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 900/1842] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:08 2022] PUT /identity/v3/projects/4b7d6516520840eea06fd2cb8888da7e/users/97857857e4524cfdbac3dfec7f953059/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:08.900585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:08.901103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:08.901191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:08.928297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] MFA Rules not processed for user `97857857e4524cfdbac3dfec7f953059`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:08.929564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:08.997151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 943/1843] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:09.001715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.064798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3bf5-e640-4ec7-ac95-992d7edaa6d3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.070377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.070996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.071283 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.071703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.072388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.074388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.104678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 901/1844] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:09 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:09.108931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.125233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553984-d5f3-4c00-844e-211a756e09b3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.183918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:09.192455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.193197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.193485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.193790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.195426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.197892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.220049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 944/1845] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:09 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:09.224040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.234781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667d54-4fb9-442c-836d-fdcaccb834ce admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.239680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.240353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.240720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.241035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.241751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.243385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.254832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 902/1846] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:09.259260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.269184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c480ae-e5a2-4ca4-a34d-91ff61fcc1ae admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.274416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.275301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.275576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.275841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] PATH_INFO: `/v3/projects/6915e93e6ff44d74902abe99ea5bd864/users/0a6dec9a64094c4a9abc64a92506fbe4/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.292653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6915e93e6ff44d74902abe99ea5bd864, user_id=0a6dec9a64094c4a9abc64a92506fbe4, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.296662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.313560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 945/1847] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:09 2022] PUT /identity/v3/projects/6915e93e6ff44d74902abe99ea5bd864/users/0a6dec9a64094c4a9abc64a92506fbe4/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:09.317552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.334768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cde10-88ea-4dbb-aed9-fdd295549ab9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.397205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.397910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.398182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.398463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.399192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.400811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7827b-a61c-4404-a3a4-cad57931bb78 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.411143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 903/1848] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:09.415444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.426335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f1ab8-b3f3-47cb-a8e2-8c19cb8fce16 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.431866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.432760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.433075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.433375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] PATH_INFO: `/v3/projects/6915e93e6ff44d74902abe99ea5bd864/users/0a6dec9a64094c4a9abc64a92506fbe4/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.435446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6915e93e6ff44d74902abe99ea5bd864, user_id=0a6dec9a64094c4a9abc64a92506fbe4, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.438150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.449415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 946/1849] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:09 2022] PUT /identity/v3/projects/6915e93e6ff44d74902abe99ea5bd864/users/0a6dec9a64094c4a9abc64a92506fbe4/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:09.461380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.461869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.462166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.494023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] MFA Rules not processed for user `0a6dec9a64094c4a9abc64a92506fbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:09.495266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:09.543279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:09.572803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 904/1850] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:09.583606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:09.585602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:09.643673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de28c32-6fa4-496e-881c-47fe4f8d53d1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:09.648078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:09.648735 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:09.649043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:09.649327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:09.659194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:09.660699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6303c-936b-436b-860e-96bb21790293 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:09.671116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 947/1851] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:11.102761 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:11.169302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b22a-9097-4ad0-8735-482d76ebe126 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:11.173856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:11.174743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:11.175031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:11.175306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] PATH_INFO: `/v3/users/97857857e4524cfdbac3dfec7f953059` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:11.176501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97857857e4524cfdbac3dfec7f953059)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:11.178535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:11.227275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97857857e4524cfdbac3dfec7f953059'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.234799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97857857e4524cfdbac3dfec7f953059'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.246412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97857857e4524cfdbac3dfec7f953059'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.247000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97857857e4524cfdbac3dfec7f953059'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.247609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97857857e4524cfdbac3dfec7f953059'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.264849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 905/1852] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:11 2022] DELETE /identity/v3/users/97857857e4524cfdbac3dfec7f953059 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:11.269514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:11.269973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:11.270074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:11.288693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:11.289927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:11.362353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 948/1853] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:11.372690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:11.374175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:11.433921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d8df9-9314-4ac3-b439-4baad1a6e59c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:11.437765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:11.438420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:11.438699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:11.438970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:11.448710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:11.450186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:11.455472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:11.461442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 906/1854] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:11.765913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:11.770296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:11.790646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d03ac-94b5-41a4-8de7-cf08ab2c5404 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:11.796565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:11.797604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:11.797883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:11.798158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] PATH_INFO: `/v3/projects/4b7d6516520840eea06fd2cb8888da7e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:11.799537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7d6516520840eea06fd2cb8888da7e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:11.801825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:11.847476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:11.851913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7d6516520840eea06fd2cb8888da7e', 'request_id': 'req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.859333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7d6516520840eea06fd2cb8888da7e', 'request_id': 'req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.865258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7d6516520840eea06fd2cb8888da7e', 'request_id': 'req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.878924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] The token cache is being invalidate because project 4b7d6516520840eea06fd2cb8888da7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:11.879188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.879800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:11.881664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 949/1855] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:11 2022] DELETE /identity/v3/projects/4b7d6516520840eea06fd2cb8888da7e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:11.886219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:11.895827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab70e-3973-4da3-a7f3-07c7f7882027 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:11.952479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:11.953413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:11.953696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:11.953968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] PATH_INFO: `/v3/users/0a6dec9a64094c4a9abc64a92506fbe4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:11.955197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6dec9a64094c4a9abc64a92506fbe4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:11.957194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.010313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6dec9a64094c4a9abc64a92506fbe4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.017603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6dec9a64094c4a9abc64a92506fbe4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.026271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6dec9a64094c4a9abc64a92506fbe4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.026916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6dec9a64094c4a9abc64a92506fbe4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.027510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-69016b6f-c5d9-4473-bc3f-416b745bdeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6dec9a64094c4a9abc64a92506fbe4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.048871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 907/1856] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:11 2022] DELETE /identity/v3/users/0a6dec9a64094c4a9abc64a92506fbe4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:12.386472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.388132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:12.402358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d0ac4-c146-4874-b74f-2d3ccd0afed6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.489150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.490050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.490330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.490657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] PATH_INFO: `/v3/projects/6915e93e6ff44d74902abe99ea5bd864` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.491854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6915e93e6ff44d74902abe99ea5bd864)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.493937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.555098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6915e93e6ff44d74902abe99ea5bd864', 'request_id': 'req-b9c94a0c-e66c-447b-9958-6f417d43a9aa'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.561883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6915e93e6ff44d74902abe99ea5bd864', 'request_id': 'req-b9c94a0c-e66c-447b-9958-6f417d43a9aa'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.568009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6915e93e6ff44d74902abe99ea5bd864', 'request_id': 'req-b9c94a0c-e66c-447b-9958-6f417d43a9aa'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.582260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] The token cache is being invalidate because project 6915e93e6ff44d74902abe99ea5bd864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:12.582582 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.583350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b9c94a0c-e66c-447b-9958-6f417d43a9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:12.585429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 950/1857] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:12 2022] DELETE /identity/v3/projects/6915e93e6ff44d74902abe99ea5bd864 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:12.590229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.590699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.590889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.609656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:12.610871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:12.685073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 908/1858] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:12.686060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.686355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.686668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.707485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:12.708999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:12.718600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 951/1859] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:12.722799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.733579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fa1be-61f6-4b1e-8e78-3856dc9475db None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.751596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.753670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.754463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.755286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.757649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:12.758311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.760198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.777383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 909/1860] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:05:12.781876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.791684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8550a0-50b6-4ac0-b8af-fd697e826913 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.796386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.797101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.797384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.797684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.798392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.800321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.834411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 952/1861] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:12 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:12.838839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.847856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.850392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:12.853648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dd6f5-1d73-4649-9058-097e562dfaa6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.857857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:12.893489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:12.922224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.922945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.923221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.923498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.924213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.926233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.928878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d728-399f-424c-b8ff-e54df2a860ee admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.933600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.934493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.934771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.935072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] PATH_INFO: `/v3/users/82ab2a3d306648259020716d3e7650ef` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.936362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ab2a3d306648259020716d3e7650ef)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.938519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.951869 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 910/1862] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:12 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:12.953365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:12.963622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56881-c376-4e81-8392-0e7e0bd54a8a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:12.969647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:12.970443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:12.970734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:12.970959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:12.971722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:12.973566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:12.991898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 911/1863] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:12.994883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:13.004219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ab2a3d306648259020716d3e7650ef'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.005361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548496b-38e7-409b-8aa2-0fe789e18a71 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:13.011947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:13.012779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.013014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.013413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.014255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:13.014946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ab2a3d306648259020716d3e7650ef'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.016424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:13.023665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ab2a3d306648259020716d3e7650ef'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.024686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ab2a3d306648259020716d3e7650ef'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.025281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8a3d08b0-81ce-4108-b673-5d29f5e29efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ab2a3d306648259020716d3e7650ef'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.028242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 912/1864] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:13.039276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:13.041217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.049225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 953/1865] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:12 2022] DELETE /identity/v3/users/82ab2a3d306648259020716d3e7650ef => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:13.053812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.054112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0bb82-3628-43ba-ac68-bf0add90b3d0 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:13.054221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.054519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.075833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:13.077164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.163819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 954/1866] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:13.173680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:13.181649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.268208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:13.268636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.269166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.269687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] PATH_INFO: `/v3/projects/9f9810b5c4034f27a3c82428fa95c593/users/37f130c078f0411fbf2f1ed952b7b640/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.291976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9810b5c4034f27a3c82428fa95c593, user_id=37f130c078f0411fbf2f1ed952b7b640, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:13.295187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be667a40-aa50-4c63-b619-dc09cb591d1c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:13.300017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203898ed-6db8-4a40-8d7e-80647b98d668 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:13.306121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:13.307171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.307840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.308318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.317157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:13.320746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:13.322576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:13.330240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 913/1867] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:13 2022] PUT /identity/v3/projects/9f9810b5c4034f27a3c82428fa95c593/users/37f130c078f0411fbf2f1ed952b7b640/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:13.332590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.334249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.334548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.335812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 955/1868] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:05:13.372208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] MFA Rules not processed for user `37f130c078f0411fbf2f1ed952b7b640`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:13.373730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.459619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:13.490891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 914/1869] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:13.498661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:13.500163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.564174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e633dc-e6ee-4953-bb59-06f9287d06ab service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:13.600440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:13.605370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:13.653289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:13.654225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.654225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.654449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.670667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:13.672145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6386ccd-419d-46ae-8b8d-4bb4f779fad1 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:13.697174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 956/1870] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:13.719158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2940c0-9377-46a3-90c3-b99bb483dd92 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:13.727818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:13.728898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.731435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.731799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] PATH_INFO: `/v3/users/f3fb076c343c42e889bea217042a78bd` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:13.733261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fb076c343c42e889bea217042a78bd)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:13.737061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:13.851456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fb076c343c42e889bea217042a78bd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.873509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fb076c343c42e889bea217042a78bd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.902112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fb076c343c42e889bea217042a78bd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.902942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fb076c343c42e889bea217042a78bd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.904020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fb076c343c42e889bea217042a78bd'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:13.992987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 915/1871] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:13 2022] DELETE /identity/v3/users/f3fb076c343c42e889bea217042a78bd => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:13.997524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:13.998258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:13.998729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:14.066502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:14.071709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:14.190588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 957/1872] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:14.242309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:14.244746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:14.410076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a627-cbdc-4be0-8c50-05b9337001bc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:14.417736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:14.418611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:14.419009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:14.419350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:14.436112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:14.440501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:14.453884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 916/1873] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:15.023730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:15.025941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.041654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1565048-f9ce-45ab-9c3f-0a8a511f3079 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:15.049125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:15.051018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.051556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.052070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] PATH_INFO: `/v3/projects/c5679f30975246b19d849ff3aca53b1e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.083440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5679f30975246b19d849ff3aca53b1e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:15.122146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:15.192576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:15.198293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.220808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5497f38-543e-40d3-98eb-be4369698171 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:15.231473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:15.233996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5679f30975246b19d849ff3aca53b1e', 'request_id': 'req-046db955-433c-4143-8c55-7476d79364fe'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.237455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:15.250457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5679f30975246b19d849ff3aca53b1e', 'request_id': 'req-046db955-433c-4143-8c55-7476d79364fe'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.262482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5679f30975246b19d849ff3aca53b1e', 'request_id': 'req-046db955-433c-4143-8c55-7476d79364fe'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.294310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] The token cache is being invalidate because project c5679f30975246b19d849ff3aca53b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:15.294641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.295303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.296948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 958/1874] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:15 2022] DELETE /identity/v3/projects/c5679f30975246b19d849ff3aca53b1e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:15.301064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:15.303735 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.312149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:15.313084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.313369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.313654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] PATH_INFO: `/v3/projects/fa2c71e4d23a4b54996f08230388a87b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.315239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2c71e4d23a4b54996f08230388a87b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:15.317816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:15.434554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2c71e4d23a4b54996f08230388a87b', 'request_id': 'req-45854035-aebc-4a90-8758-b47b6139bc08'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.447334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046db955-433c-4143-8c55-7476d79364fe admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:15.458549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2c71e4d23a4b54996f08230388a87b', 'request_id': 'req-45854035-aebc-4a90-8758-b47b6139bc08'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.489186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2c71e4d23a4b54996f08230388a87b', 'request_id': 'req-45854035-aebc-4a90-8758-b47b6139bc08'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.523165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] The token cache is being invalidate because project fa2c71e4d23a4b54996f08230388a87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:15.523600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.525265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-45854035-aebc-4a90-8758-b47b6139bc08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.528627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 917/1875] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:15 2022] DELETE /identity/v3/projects/fa2c71e4d23a4b54996f08230388a87b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:15.533498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c3e1-dd28-4330-8e06-c6e262e9a828 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.533860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c3e1-dd28-4330-8e06-c6e262e9a828 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.534239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3c3e1-dd28-4330-8e06-c6e262e9a828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.555788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:15.556941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.557215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.558499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] PATH_INFO: `/v3/users/700d2ae4d2ae416b8f26f24997c839bd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.560040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=700d2ae4d2ae416b8f26f24997c839bd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:15.562210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:15.590074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fbf3c3e1-dd28-4330-8e06-c6e262e9a828 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:15.592668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fbf3c3e1-dd28-4330-8e06-c6e262e9a828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.629098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 918/1876] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:15.631172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.631693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.631958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.663341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700d2ae4d2ae416b8f26f24997c839bd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.685755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:15.687004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.691301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700d2ae4d2ae416b8f26f24997c839bd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.701396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700d2ae4d2ae416b8f26f24997c839bd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.702082 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700d2ae4d2ae416b8f26f24997c839bd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.702696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700d2ae4d2ae416b8f26f24997c839bd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:15.711854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 919/1877] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:15.714762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:15.749234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e0428-8fbb-4536-af9b-4f6fdbbbf051 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:15.772494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 959/1878] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:15 2022] DELETE /identity/v3/users/700d2ae4d2ae416b8f26f24997c839bd => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:15.773252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:15.774039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.774322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.774652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.775390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:15.777236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:15.781369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:15.801902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:15.803538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:15.809251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 920/1879] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:15.811605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:15.931596 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064148d-0e88-44df-a83b-4ed584403415 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:15.948250 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:15.951077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:15.952534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:15.952929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:15.955192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:15.960349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1792a6-636d-4b97-b596-933f87795e32 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.020983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 960/1880] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:15 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:16.024182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.037662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20ec8-0720-4529-b54c-6bd18c62ed63 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.142190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c95fa-fa8a-4676-aed1-2616e32ab258 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.142880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c95fa-fa8a-4676-aed1-2616e32ab258 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.143382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c95fa-fa8a-4676-aed1-2616e32ab258 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.144071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.144884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.145283 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.145503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 961/1881] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 28 12:05:16.146143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.147010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.149393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.162391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.162788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.163530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.182542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 921/1882] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:16.183802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.187951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:16.189438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:16.191688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.201714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078cc1b-399f-445f-b263-3c766b666fef admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.215923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.216628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.217457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.217735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.218483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.220126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.247220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 922/1883] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:16.248432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.265735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01de04-8c72-43ac-92fe-9fa3f0a09c34 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.270675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.271660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.272136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.272560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.273688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.275617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.290276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 923/1884] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:16.294702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.296812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 962/1885] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:16.312481 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.314055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.314942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b414ade-3b4b-49cb-9f34-7b36522f2a1f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.324665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.326250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.326799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.327626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] PATH_INFO: `/v3/projects/dd3d5f5638614d5c9af6fea9d8b1da16/users/bfc0deab09db4d47b8ad47b8540f7ad1/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.335685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e604f-cb3e-4522-b20f-84b67cc7c8a4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.343874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.345260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.345559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.348205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.348529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3d5f5638614d5c9af6fea9d8b1da16, user_id=bfc0deab09db4d47b8ad47b8540f7ad1, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.352571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.364152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.367191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e697-38d7-41a6-a076-f7bdae87fc74 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.378570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 924/1886] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:16 2022] PUT /identity/v3/projects/dd3d5f5638614d5c9af6fea9d8b1da16/users/bfc0deab09db4d47b8ad47b8540f7ad1/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:16.379556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.380990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 963/1887] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:05:16.391782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.392077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.392363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.393506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.436852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] MFA Rules not processed for user `bfc0deab09db4d47b8ad47b8540f7ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:16.439465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.483901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a30842-9737-4b7a-af4c-f5a84825bacf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.488652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.489547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.489828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.490098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] PATH_INFO: `/v3/projects/4f3081e7a82b478790396813d4240d3b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.491332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3081e7a82b478790396813d4240d3b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.493290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.514616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:16.521511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 964/1888] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:16.533854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.536067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.550450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3081e7a82b478790396813d4240d3b', 'request_id': 'req-efb69266-59a1-4d27-89a4-2607acee93ab'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:16.570485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3081e7a82b478790396813d4240d3b', 'request_id': 'req-efb69266-59a1-4d27-89a4-2607acee93ab'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:16.587449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3081e7a82b478790396813d4240d3b', 'request_id': 'req-efb69266-59a1-4d27-89a4-2607acee93ab'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:16.603984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206e6b-93d3-4dc1-b9a0-0f312baf7d0d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.604923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] The token cache is being invalidate because project 4f3081e7a82b478790396813d4240d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:16.605236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:16.605888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-efb69266-59a1-4d27-89a4-2607acee93ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:16.607795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 925/1889] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:16 2022] DELETE /identity/v3/projects/4f3081e7a82b478790396813d4240d3b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:16.612384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4307275-c62d-48df-ad19-2b6b6d0161f4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.612669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4307275-c62d-48df-ad19-2b6b6d0161f4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.612974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4307275-c62d-48df-ad19-2b6b6d0161f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.632507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c4307275-c62d-48df-ad19-2b6b6d0161f4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:16.633755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c4307275-c62d-48df-ad19-2b6b6d0161f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.673765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.674511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.675010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.675363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.677688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.690118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:16.711178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 926/1890] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:16.719091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.719373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.719650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.738563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:16.739768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:16.744577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.746138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.749114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 927/1891] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:16.752584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.756227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 965/1892] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:16.773397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347ee34-3907-4558-8758-2a0d3018dcc4 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.781765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.782718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.783222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.783701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.784750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:16.785287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.787355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.802861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 928/1893] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:16.809766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.834236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916e1-9bc7-4a2f-adb5-aa774acaccc2 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.842377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.843090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.843373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.843650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.844352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.846361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.875723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 966/1894] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:16.880077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:16.891099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153eebd-bc21-48ee-9c0d-3f712a974add admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:16.949940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:16.950625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:16.950906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:16.951178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:16.951875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:16.953870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:16.981454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 929/1895] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:16 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:16.982705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.013329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffac4f3-e86c-4a7a-8856-0b158fbb7975 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.035813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.036527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.036813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.037120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.038186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.043843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.075161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 967/1896] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:17.083239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.097363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958b95b-6c34-499c-a9f2-dabfa3131345 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.114115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.116155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.116546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.116846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.118159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.121540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebad7b-3786-4973-a3cc-acd469f43737 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.150939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 930/1897] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:17.156539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.170928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c03a4-65a4-49fe-85c9-a0bbedcfed72 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.183457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.187998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.188619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.189269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] PATH_INFO: `/v3/projects/e5e732dc47424b84bc3e7c0137d6e3b7/users/93160323e8d24c9483d106624e49293a/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.224014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e732dc47424b84bc3e7c0137d6e3b7, user_id=93160323e8d24c9483d106624e49293a, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.227617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.256125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 968/1898] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:17 2022] PUT /identity/v3/projects/e5e732dc47424b84bc3e7c0137d6e3b7/users/93160323e8d24c9483d106624e49293a/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:17.267658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9395-e935-47b9-881e-49daf421faee None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.268169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9395-e935-47b9-881e-49daf421faee None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.268267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9395-e935-47b9-881e-49daf421faee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.341101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-466a9395-e935-47b9-881e-49daf421faee None None] MFA Rules not processed for user `93160323e8d24c9483d106624e49293a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:17.343229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-466a9395-e935-47b9-881e-49daf421faee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:17.443213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 931/1899] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:17.448173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.466173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:17.578679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d90e2-feb8-410e-9e66-cfae2e276ee1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.587319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.589128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.589562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.589992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.590887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.594558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.642443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 969/1900] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:17 2022] POST /identity/v3/projects => generated 389 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:17.647232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a9395-e935-47b9-881e-49daf421faee None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.682309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a9395-e935-47b9-881e-49daf421faee None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.764663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:17.813409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.815165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.815373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.815575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.816353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.819207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.867980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.874770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 932/1901] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:17 2022] POST /identity/v3/users => generated 301 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:17.881806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3201513-1eb1-4fee-a8ff-f065aae7c98a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.888136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.888471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.889166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.889166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.889772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.891423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.923113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:17.923786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 970/1902] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:17.940171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc3a27-8542-4327-a22b-395825703224 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:17.952004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:17.954201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:17.954491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:17.954825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] PATH_INFO: `/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3/users/74d5ff0c4a5249559bd7c779077e2e7d/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:17.978567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813aa5426adf4ccf82ad8037dd5ecfd3, user_id=74d5ff0c4a5249559bd7c779077e2e7d, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:17.981322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:17.996939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 933/1903] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:17 2022] PUT /identity/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3/users/74d5ff0c4a5249559bd7c779077e2e7d/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:18.000880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:18.012477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beda4c7-ccdd-4daf-abc6-37323d3fc837 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:18.093752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:18.094579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.094842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.095073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:18.096033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:18.097779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efcaa7-f157-4eb1-b5ee-4491a63db58b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:18.112393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 971/1904] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:18.113664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:18.128754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9ac89-2e13-4be6-8328-a2b93c916581 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:18.140133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:18.148282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.148636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.149029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] PATH_INFO: `/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3/users/74d5ff0c4a5249559bd7c779077e2e7d/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:18.151926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813aa5426adf4ccf82ad8037dd5ecfd3, user_id=74d5ff0c4a5249559bd7c779077e2e7d, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:18.159051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:18.184772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 934/1905] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:18 2022] PUT /identity/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3/users/74d5ff0c4a5249559bd7c779077e2e7d/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:18.186815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.187303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.187395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:18.222657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] MFA Rules not processed for user `74d5ff0c4a5249559bd7c779077e2e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:18.224212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:18.322260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 972/1906] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:18.338951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:18.348008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:18.470150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e82dd2-04b5-4c0a-9ca2-7dbd1a686886 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:18.474236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:18.475068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.475472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.475753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:18.486878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:18.488337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6345b9-aa2e-4a02-9bc9-792aa1b4ca83 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:18.500796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 935/1907] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:18.637746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:18.650131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:18.713999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e929e3-365d-4f1d-98f2-0e37cafdba1e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:18.733898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:18.737630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.738014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.738368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] PATH_INFO: `/v3/users/93160323e8d24c9483d106624e49293a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:18.742884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93160323e8d24c9483d106624e49293a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:18.746886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:18.882765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93160323e8d24c9483d106624e49293a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:18.905106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93160323e8d24c9483d106624e49293a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:18.928972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93160323e8d24c9483d106624e49293a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:18.931634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93160323e8d24c9483d106624e49293a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:18.934680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93160323e8d24c9483d106624e49293a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:18.982120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 973/1908] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:18 2022] DELETE /identity/v3/users/93160323e8d24c9483d106624e49293a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:18.988718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:18.988718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:18.989001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:19.010056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:19.011645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:19.118148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 936/1909] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:19.127968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:19.130021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:19.193727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921231-daf8-4bff-b04d-bc292a36fa05 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:19.197474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:19.198234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:19.198524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:19.198798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:19.208138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:19.209733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:19.219954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 974/1910] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:19.538419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:19.540639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:19.554338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ec0bf-f0b8-4acb-9a65-f8d8c9173073 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:19.559370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:19.560375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:19.560659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:19.560960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] PATH_INFO: `/v3/projects/e5e732dc47424b84bc3e7c0137d6e3b7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:19.562176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e732dc47424b84bc3e7c0137d6e3b7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:19.564117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:19.597141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:19.618773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e732dc47424b84bc3e7c0137d6e3b7', 'request_id': 'req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.625555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e732dc47424b84bc3e7c0137d6e3b7', 'request_id': 'req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.631153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e732dc47424b84bc3e7c0137d6e3b7', 'request_id': 'req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.644512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] The token cache is being invalidate because project e5e732dc47424b84bc3e7c0137d6e3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:19.644793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.645465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.647625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 937/1911] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:19 2022] DELETE /identity/v3/projects/e5e732dc47424b84bc3e7c0137d6e3b7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:19.651928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:19.661726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8989ce-92d6-470a-a234-04df4d0f5d71 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:19.689048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:19.722508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:19.723399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:19.723681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:19.723955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] PATH_INFO: `/v3/users/74d5ff0c4a5249559bd7c779077e2e7d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:19.725415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d5ff0c4a5249559bd7c779077e2e7d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:19.727410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:19.775775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d5ff0c4a5249559bd7c779077e2e7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.783389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d5ff0c4a5249559bd7c779077e2e7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.790772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d5ff0c4a5249559bd7c779077e2e7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.791412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d5ff0c4a5249559bd7c779077e2e7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.792006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b340c6ca-b820-45e7-ae27-6d593a0caae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d5ff0c4a5249559bd7c779077e2e7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:19.810150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 975/1912] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:19 2022] DELETE /identity/v3/users/74d5ff0c4a5249559bd7c779077e2e7d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:20.427425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:20.429358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:20.444968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e1bd-f89b-48ce-adff-fc3acf0ab09b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:20.529980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:20.532369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.533009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.533526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] PATH_INFO: `/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.535146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813aa5426adf4ccf82ad8037dd5ecfd3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:20.542271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:20.598690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813aa5426adf4ccf82ad8037dd5ecfd3', 'request_id': 'req-053a91b2-f55e-4992-ab36-d4fcadeb3392'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:20.605767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813aa5426adf4ccf82ad8037dd5ecfd3', 'request_id': 'req-053a91b2-f55e-4992-ab36-d4fcadeb3392'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:20.611259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813aa5426adf4ccf82ad8037dd5ecfd3', 'request_id': 'req-053a91b2-f55e-4992-ab36-d4fcadeb3392'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:20.624862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] The token cache is being invalidate because project 813aa5426adf4ccf82ad8037dd5ecfd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:20.625176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:20.625811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-053a91b2-f55e-4992-ab36-d4fcadeb3392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:20.627437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 938/1913] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:20 2022] DELETE /identity/v3/projects/813aa5426adf4ccf82ad8037dd5ecfd3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:20.632524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8546a-9245-43cb-8416-843537404767 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.633033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8546a-9245-43cb-8416-843537404767 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.633141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8546a-9245-43cb-8416-843537404767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.651570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2aa8546a-9245-43cb-8416-843537404767 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:20.652726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2aa8546a-9245-43cb-8416-843537404767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:20.715980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 976/1914] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:20.725349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.725729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.726114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.747157 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:20.748371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:20.757959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 939/1915] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:20.762785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8546a-9245-43cb-8416-843537404767 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:20.774211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8546a-9245-43cb-8416-843537404767 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:20.779413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:20.780092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.780375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.780645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.781421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:20.781727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:20.783346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:20.797050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 977/1916] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:20.801401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:20.810629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e79e4a-bb5e-4a46-837b-12f0b7c907bf None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:20.815419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:20.816074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.816353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.816623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.817332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:20.819528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:20.847179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 940/1917] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:20.851513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:20.861927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9454a0-a8b7-4583-b42d-bbca3f2b47b4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:20.895827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:20.924018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:20.924625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.924943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.925226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.925924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:20.927904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:20.949231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 978/1918] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:20 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:20.954970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:20.966290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa477c14-2f3b-49d3-be10-3a75bf8ee104 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:20.971839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:20.972504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:20.972805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:20.973102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:20.973824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:20.975975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:20.986916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 941/1919] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:20.991143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:21.001937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750afc08-2518-490f-a6bb-316e6b41c290 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:21.007433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:21.008127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:21.008407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:21.008668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:21.009413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:21.011118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b32c2-5f6e-4bd9-aca3-e559f4a18eca admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:21.026627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 979/1920] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:21.027648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:21.034404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:21.038526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831875b6-27b0-49d8-bcdf-7e1b32d69866 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:21.045590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:21.046867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:21.047315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:21.047735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] PATH_INFO: `/v3/projects/fcdabcb940cc4b9a96d8f5f0f2a0d53c/users/5a8f14328981453da857c662c038c6d9/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:21.063475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdabcb940cc4b9a96d8f5f0f2a0d53c, user_id=5a8f14328981453da857c662c038c6d9, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:21.066171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:21.081232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 942/1921] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:21 2022] PUT /identity/v3/projects/fcdabcb940cc4b9a96d8f5f0f2a0d53c/users/5a8f14328981453da857c662c038c6d9/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:21.092543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:21.092865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:21.093166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:21.120099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] MFA Rules not processed for user `5a8f14328981453da857c662c038c6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:21.121383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:21.188827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 980/1922] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:21.199785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:21.201699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:21.260412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d24cc-bf78-4490-940f-a385c693e988 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:21.264343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:21.265023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:21.265310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:21.265584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:21.274642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:21.276125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dacc57-b722-4e01-a1d8-9be5b14abaf0 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:21.285131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 943/1923] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:28.594627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:28.596192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:28.676181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392c6c4-41af-4a8b-b983-e4878b69aa32 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:28.681303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:28.682274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:28.682621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:28.683096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] PATH_INFO: `/v3/users/93cb5dcef86246ada2d7ee4a08b0d733` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:28.684246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93cb5dcef86246ada2d7ee4a08b0d733)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:28.686325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:28.778956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:28.781400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93cb5dcef86246ada2d7ee4a08b0d733'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:28.807118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93cb5dcef86246ada2d7ee4a08b0d733'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:28.827068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93cb5dcef86246ada2d7ee4a08b0d733'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:28.827927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93cb5dcef86246ada2d7ee4a08b0d733'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:28.828693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93cb5dcef86246ada2d7ee4a08b0d733'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:28.851727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 981/1924] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:28 2022] DELETE /identity/v3/users/93cb5dcef86246ada2d7ee4a08b0d733 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:28.856401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:28.856971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:28.856971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:28.876994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:28.878752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:28.925610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:28.926854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:28.978899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 944/1925] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:28.987178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:28.989714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:29.105762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1de362-e473-48b8-a505-47bb7ecdea1e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:29.124837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:29.126457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:29.128095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:29.128488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:29.158873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b02b0-cfa4-42d3-a88a-ae2ff865cfc7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:29.167176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:29.172285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39f3c5-a6f5-4b26-8846-21d2bd1c28bf service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:29.175979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:29.180356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:29.180773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:29.181161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] PATH_INFO: `/v3/users/5a8f14328981453da857c662c038c6d9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:29.186079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8f14328981453da857c662c038c6d9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:29.189073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:29.201404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 945/1926] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:29.258744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8f14328981453da857c662c038c6d9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:29.267604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8f14328981453da857c662c038c6d9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:29.276597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8f14328981453da857c662c038c6d9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:29.278021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8f14328981453da857c662c038c6d9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:29.278674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8f14328981453da857c662c038c6d9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:29.301925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 982/1927] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:28 2022] DELETE /identity/v3/users/5a8f14328981453da857c662c038c6d9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:29.306963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:29.307272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:29.307482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:29.326685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:29.328203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:29.354575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:29.412866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 946/1928] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:29.426294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:29.427793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:29.494308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:29.526690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025f913-902b-40d4-b5b6-569e7129b075 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:29.531722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:29.532874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:29.533426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:29.534156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:29.548003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:29.550228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:29.567092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 983/1929] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:30.256290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:30.258236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:30.270967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd764a-873b-4622-ae05-e79fa8b8c1e1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:30.277630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:30.278633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.278788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.279066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] PATH_INFO: `/v3/projects/bc6cfc425cc74fdfaa65a7a3cf60ffb4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.280410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6cfc425cc74fdfaa65a7a3cf60ffb4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:30.282505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:30.345800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6cfc425cc74fdfaa65a7a3cf60ffb4', 'request_id': 'req-a34e8ed6-d111-417b-9d62-d9faafef27a6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.355341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6cfc425cc74fdfaa65a7a3cf60ffb4', 'request_id': 'req-a34e8ed6-d111-417b-9d62-d9faafef27a6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.361572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6cfc425cc74fdfaa65a7a3cf60ffb4', 'request_id': 'req-a34e8ed6-d111-417b-9d62-d9faafef27a6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.375598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] The token cache is being invalidate because project bc6cfc425cc74fdfaa65a7a3cf60ffb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:30.375787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.376410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.378160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 947/1930] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:30 2022] DELETE /identity/v3/projects/bc6cfc425cc74fdfaa65a7a3cf60ffb4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:30.382239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:30.392560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e097a12-dd82-4cd1-8ff7-b9d7ea02592c service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:30.425579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:30.427032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:30.462208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:30.463493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.463493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.463852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] PATH_INFO: `/v3/users/f470c32b1525402f8a177e645ad4f58a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.465251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f470c32b1525402f8a177e645ad4f58a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:30.467744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:30.528874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f470c32b1525402f8a177e645ad4f58a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.538435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e8ed6-d111-417b-9d62-d9faafef27a6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:30.538946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f470c32b1525402f8a177e645ad4f58a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.544375 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:30.545311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.545592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.545862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] PATH_INFO: `/v3/projects/fcdabcb940cc4b9a96d8f5f0f2a0d53c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.547158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdabcb940cc4b9a96d8f5f0f2a0d53c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:30.548640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f470c32b1525402f8a177e645ad4f58a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.550372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:30.550506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f470c32b1525402f8a177e645ad4f58a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.550506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9c1f5181-1fd0-4417-83a2-42f1de6c8ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f470c32b1525402f8a177e645ad4f58a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.580721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 984/1931] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:30 2022] DELETE /identity/v3/users/f470c32b1525402f8a177e645ad4f58a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:30.633252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdabcb940cc4b9a96d8f5f0f2a0d53c', 'request_id': 'req-f6487caa-40ba-44c9-97e7-feeafd68201b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.636357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:30.650074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdabcb940cc4b9a96d8f5f0f2a0d53c', 'request_id': 'req-f6487caa-40ba-44c9-97e7-feeafd68201b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.661625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdabcb940cc4b9a96d8f5f0f2a0d53c', 'request_id': 'req-f6487caa-40ba-44c9-97e7-feeafd68201b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.697293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] The token cache is being invalidate because project fcdabcb940cc4b9a96d8f5f0f2a0d53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:30.700229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.701651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6487caa-40ba-44c9-97e7-feeafd68201b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:30.704248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 948/1932] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:30 2022] DELETE /identity/v3/projects/fcdabcb940cc4b9a96d8f5f0f2a0d53c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:30.710416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.710708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.710987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.733543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:30.734958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:30.836229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 985/1933] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:30.847125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.847125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.847125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.889432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:30.890720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:30.901729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 949/1934] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:30.905867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:30.914610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:30.920681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7009f7-e862-4f8a-abfe-982a678b2901 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:30.929394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:30.930218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.930507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.930935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.931775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:30.932205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:30.934727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:30.960084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 986/1935] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:30.960960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:30.976114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4794124-57fe-4cad-aaae-f2c347c69934 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:30.983452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:30.985295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:30.986084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:30.986084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:30.986609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:30.988705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.058045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 950/1936] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:30 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:31.060298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.073680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c8c6c-272b-4b87-8c24-04e98090b47b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.157596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.159522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:31.219254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.220083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.220464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.220814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.221890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.224772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.259469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 987/1937] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:31.264728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.283212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561903-3508-4255-9328-00ba118958e8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.287185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.288568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.288688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.289636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.290398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.292039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.303586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 988/1938] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:31.308021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.318751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07197e86-8d1a-45a0-ba06-8f532cbfd3d4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.321717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485aa5e6-9003-467a-93d0-cfb6a118e74d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.326971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.327886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.328011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.328289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.328568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] PATH_INFO: `/v3/projects/f877a302f0f44c77a0dd65ef2487bca7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.328800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.329413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.329761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.330030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f877a302f0f44c77a0dd65ef2487bca7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.330568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.331995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.332479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.348727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 989/1939] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:31.353667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.368270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030060b-142d-4ace-8d4c-2de0b27ef1da admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.373528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.374441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.374782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.375093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] PATH_INFO: `/v3/projects/f5a0c527e3434d9d9497960e10b2072f/users/56fd4fa666e149948a381671063723fa/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.394498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a0c527e3434d9d9497960e10b2072f, user_id=56fd4fa666e149948a381671063723fa, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.397177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2827106-57f9-4b94-9b73-15346d0d6e68 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.407868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f877a302f0f44c77a0dd65ef2487bca7', 'request_id': 'req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:31.415741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 990/1940] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:31 2022] PUT /identity/v3/projects/f5a0c527e3434d9d9497960e10b2072f/users/56fd4fa666e149948a381671063723fa/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:31.419625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f877a302f0f44c77a0dd65ef2487bca7', 'request_id': 'req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:31.426189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096142b-3542-46c2-8aed-b206dfef0787 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.426473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096142b-3542-46c2-8aed-b206dfef0787 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.426751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096142b-3542-46c2-8aed-b206dfef0787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.429451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f877a302f0f44c77a0dd65ef2487bca7', 'request_id': 'req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:31.448071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] The token cache is being invalidate because project f877a302f0f44c77a0dd65ef2487bca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:31.448361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:31.449586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4b115bd9-c543-429e-b9f8-e2eb05b7bfb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:31.451053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 951/1941] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:31 2022] DELETE /identity/v3/projects/f877a302f0f44c77a0dd65ef2487bca7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:31.455545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.455834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.456108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.465440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1096142b-3542-46c2-8aed-b206dfef0787 None None] MFA Rules not processed for user `56fd4fa666e149948a381671063723fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:31.466651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1096142b-3542-46c2-8aed-b206dfef0787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:31.475518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:31.476750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:31.516847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:31.535133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 991/1942] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:31.540373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 952/1943] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:31.547565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.548096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.548390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.548668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.549200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:31.571199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:31.572447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:31.581215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 992/1944] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:31.586141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.595589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fecbba-ec18-4f73-9760-eef0f2ee112e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.600767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.601317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.601606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.601884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.602619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:31.602927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.604536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.614408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42e2b-1247-4e12-9ee2-f54730c178ec None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.619247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 993/1945] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:05:31.619396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.619396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.619396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.619661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.620808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.626807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:31.628448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.630007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.630687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7e0b-1f2e-4f5b-8ce2-f0d4b8253a2a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.636753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.637496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.637841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.638197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.638941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.639158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 953/1946] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:31.641072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.676274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 994/1947] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:31.677700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.689340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfe212-3190-4a94-9caa-68f0b06db987 service glance] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.770009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.770651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.770932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.771319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.771921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.774066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.803888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 954/1948] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:31.807665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.820188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648bc54-0232-4d78-b838-061c4b87f57a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.827627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.828323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.828649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.828950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.829736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.833379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.851870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 995/1949] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:31.856633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.869974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50233-d036-42ee-b5ff-060428bcd125 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.875439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.876533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.876533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.876854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.877688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.879388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bf5da-5c56-41a8-bfae-f22d0345ee9a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.894103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 955/1950] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:31.898129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:31.909176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa86b7e-7c95-4b94-a1f4-1c8f12d0d3ca admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:31.915462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:31.916860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.917296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.917619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] PATH_INFO: `/v3/projects/f5bff2c6fc3b4d57aaae23a58e6cfd37/users/554e9190e72649f79c2451a04cc17fd7/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.933962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bff2c6fc3b4d57aaae23a58e6cfd37, user_id=554e9190e72649f79c2451a04cc17fd7, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:31.936665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:31.950983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 996/1951] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:31 2022] PUT /identity/v3/projects/f5bff2c6fc3b4d57aaae23a58e6cfd37/users/554e9190e72649f79c2451a04cc17fd7/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:31.962634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:31.962924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:31.963221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:31.995349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] MFA Rules not processed for user `554e9190e72649f79c2451a04cc17fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:31.996553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.070055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:32.082764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 956/1952] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:32.089262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.172083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dc4fe-3a2c-4d77-b231-f5c70ceeb823 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.192952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.194341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.194793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.195680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.196628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.198986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.238389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 997/1953] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:32.242397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.256872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00878ed2-9f8c-47d7-a86e-ed192759d1c5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.310811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.313127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.333052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.333756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.334131 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.334600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.336693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.338645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.364026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 957/1954] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:32.368657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.387763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d965a-c846-4ff5-bf5a-579933947581 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.392572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.393181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276185-1ac5-496c-b373-30834fea32f9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.393321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.393548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.393823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.394556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.396177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.399412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.400313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.400567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.400843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] PATH_INFO: `/v3/users/bfc0deab09db4d47b8ad47b8540f7ad1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.402194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc0deab09db4d47b8ad47b8540f7ad1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.404143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.406877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 958/1955] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:32.410902 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.421707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b631-6306-4d58-92bb-c625b874151b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.427650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.428694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.429030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.429383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.430439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.432256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.445962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 959/1956] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:32.449854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.451855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:32.465566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381a771-e537-46f0-ba87-251bb156b5a2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.468633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc0deab09db4d47b8ad47b8540f7ad1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:32.471962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.473252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.473793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.474418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] PATH_INFO: `/v3/projects/4c581d827376466196715f750334af04/users/d1e71e70fb4e4b1c8d9775138ecbacf6/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.479617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc0deab09db4d47b8ad47b8540f7ad1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:32.487976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc0deab09db4d47b8ad47b8540f7ad1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:32.488894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc0deab09db4d47b8ad47b8540f7ad1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:32.489748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca817a14-884c-4f9e-991f-c4a3aba9a2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc0deab09db4d47b8ad47b8540f7ad1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:32.494657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c581d827376466196715f750334af04, user_id=d1e71e70fb4e4b1c8d9775138ecbacf6, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.498110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fe304-d8a0-4c1b-83bb-0a519f06d3bf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.517365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 960/1957] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:32 2022] PUT /identity/v3/projects/4c581d827376466196715f750334af04/users/d1e71e70fb4e4b1c8d9775138ecbacf6/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:32.523469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.523781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 998/1958] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:32 2022] DELETE /identity/v3/users/bfc0deab09db4d47b8ad47b8540f7ad1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:32.523896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.524066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.527904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.528240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.528462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.547505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:32.548734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.557845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] MFA Rules not processed for user `d1e71e70fb4e4b1c8d9775138ecbacf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:32.559077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.621440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 961/1959] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:32.632441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.634720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.685167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 999/1960] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:32.685167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.689681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:32.723671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3051989-8b8b-47b3-8e0e-eb4f95a46238 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.729148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.730193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:32.731584 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.731965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.732351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.734028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.736229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.749860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b19e8-c316-4329-aea7-0affc7606b1a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.761632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.763371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.763884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.764230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.787985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.789649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.796071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1000/1961] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:32.801183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.809113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 962/1962] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:05:32.886116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5380a9-6f64-447d-b5c0-36b0eed55d5d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.891221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.892094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.892373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.892772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.893527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.895471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.922059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1001/1963] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:32 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:32.926316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.936838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f963eca-4057-4016-84cf-61bbda256f06 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.941707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.942376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.942819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.942945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.943634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:32.945946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:32.968395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 963/1964] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:32.972919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:32.983931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8d462-97ce-4f2c-8dce-871dc1dc3fc5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:32.989107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:32.990024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:32.990304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:32.990585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] PATH_INFO: `/v3/projects/f69b12a6089746d4a9eec6e17af64704/users/673f68f7fcfb4a4ebd024e0f5594a608/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:32.994707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:33.011741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69b12a6089746d4a9eec6e17af64704, user_id=673f68f7fcfb4a4ebd024e0f5594a608, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.017601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.044359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1002/1965] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:32 2022] PUT /identity/v3/projects/f69b12a6089746d4a9eec6e17af64704/users/673f68f7fcfb4a4ebd024e0f5594a608/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:33.049612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.070143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336c3d7-6858-43cc-8418-3e4a940db01f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.129637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.131598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.143565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.144743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.145323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.145781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.146753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.148751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.161370 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 964/1966] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:33.165895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.179374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ce0a1-d103-4018-b64a-984f01098bea admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.184369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.185411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.185978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.186327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] PATH_INFO: `/v3/projects/f69b12a6089746d4a9eec6e17af64704/users/673f68f7fcfb4a4ebd024e0f5594a608/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.188460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69b12a6089746d4a9eec6e17af64704, user_id=673f68f7fcfb4a4ebd024e0f5594a608, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.191154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde9e2c-328c-4a48-a0bf-cf50e11c2c2b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.203428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 965/1967] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:33 2022] PUT /identity/v3/projects/f69b12a6089746d4a9eec6e17af64704/users/673f68f7fcfb4a4ebd024e0f5594a608/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:33.214748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.215034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.215310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.218535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ee98c-2554-4c8d-88aa-20a325849d47 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.242926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] MFA Rules not processed for user `673f68f7fcfb4a4ebd024e0f5594a608`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:33.244139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.282554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.283799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.284097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.284458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] PATH_INFO: `/v3/projects/dd3d5f5638614d5c9af6fea9d8b1da16` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.285764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3d5f5638614d5c9af6fea9d8b1da16)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.288775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.366422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3d5f5638614d5c9af6fea9d8b1da16', 'request_id': 'req-c681627e-d41c-45b1-afd7-ec9ba17062ca'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:33.366872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 966/1968] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:33.374515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3d5f5638614d5c9af6fea9d8b1da16', 'request_id': 'req-c681627e-d41c-45b1-afd7-ec9ba17062ca'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:33.380768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3d5f5638614d5c9af6fea9d8b1da16', 'request_id': 'req-c681627e-d41c-45b1-afd7-ec9ba17062ca'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:33.393353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.395575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.418827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] The token cache is being invalidate because project dd3d5f5638614d5c9af6fea9d8b1da16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:33.421308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:33.422896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c681627e-d41c-45b1-afd7-ec9ba17062ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:33.425086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1003/1969] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:33 2022] DELETE /identity/v3/projects/dd3d5f5638614d5c9af6fea9d8b1da16 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:33.430796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b133-4d50-4246-805d-d779e95c13d5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.431138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b133-4d50-4246-805d-d779e95c13d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.431491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b133-4d50-4246-805d-d779e95c13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.458126 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-31f5b133-4d50-4246-805d-d779e95c13d5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:33.462302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:33.464702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-31f5b133-4d50-4246-805d-d779e95c13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.480165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3c36a-0025-4b3e-9aa4-19e4485cf41f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.500682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:33.546816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.547507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.547785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.548054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.549968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.568952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1004/1970] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:33.577872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.578200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.578474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.620604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:33.620604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:33.620604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1005/1971] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:33.624835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.638389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.639206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf38cc-e7f7-4b6c-9330-10d9ed2c231f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.640328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.645560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.646292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.646589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.646859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.647937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:33.648249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.649891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.651733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 967/1972] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:33.666048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1006/1973] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:05:33.667639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.683381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b87d2-3fdc-4b46-ac12-fd27a2f2b3d6 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.689857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.690532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.690814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.691090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.691799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.693779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.727702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 968/1974] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:33 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:33.731918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.742770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bbd86-ce39-47c6-b155-1dcceb9cae0a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.857093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.857093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.857093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.857354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.857872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.860195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.891955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1007/1975] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:33 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:05:33.896168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.914225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37b7ed-1aa7-4cc1-9f50-63822f70d5d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.922366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.923065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.923350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.923877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.924627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.935250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.939739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 969/1976] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:33.944999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.956680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49d7fe-2c39-4d2f-8efa-d0e52ac63e72 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:33.961549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:33.962240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:33.962567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:33.962793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:33.964168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:33.965399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbaaf56-b9ca-4614-8f58-f8e9db39a5c8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:33.980992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1008/1977] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:33.981965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:33.994355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd00653-f773-4ec3-b35a-1a510d9e9e1a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:34.001321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:34.002230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:34.002507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:34.002806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] PATH_INFO: `/v3/projects/7235a9c7eff24e0e94900ec2abc92ae5/users/9f56095d6bac4f108a73790ac9004f3b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:34.019901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7235a9c7eff24e0e94900ec2abc92ae5, user_id=9f56095d6bac4f108a73790ac9004f3b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:34.022944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:34.040937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 970/1978] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:33 2022] PUT /identity/v3/projects/7235a9c7eff24e0e94900ec2abc92ae5/users/9f56095d6bac4f108a73790ac9004f3b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:34.046316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:34.047040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:34.047499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:34.069388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:34.070961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:34.153702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1009/1979] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:34.163564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:34.165425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:34.222957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:34.234207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66010a-fc79-4063-8a82-002e742e490b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:34.238255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:34.238922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:34.239203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:34.239477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:34.251256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:34.253616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d69bb1-e867-447e-8ad7-e5958a91090f service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:34.276115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 971/1980] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:34.366954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:34.368705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:34.402409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:34.444244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf93a7-9504-46b7-b7d2-c46191ee1dc1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:34.449041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:34.449911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:34.451771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:34.452169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:34.455552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:34.562486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:34.564124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:34.586689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1010/1981] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:39.595288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:39.595814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:39.595814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:39.625702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] MFA Rules not processed for user `9f56095d6bac4f108a73790ac9004f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:39.627133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:39.718905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 972/1982] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:39.723507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:39.736200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371e439-44c1-47ae-aaf5-b700d47c2469 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:39.741290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:39.742051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:39.742400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:39.742736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:39.743650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:39.746188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:39.797451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1011/1983] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:39 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:39.798764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:39.811387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2141492-bfa4-4344-bc8c-1c17914e709d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:39.918116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:39.918942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:39.919226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:39.919500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:39.920224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:39.922243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:39.945186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 973/1984] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:39 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:39.949480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:39.968321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b9a2-3081-42b6-a916-1ee57cdadff4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:39.973952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:39.974781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:39.975201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:39.975531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:39.976215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:39.978293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:40.003280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1012/1985] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:40.005126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:40.017137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544845-e9b6-41f6-a912-497f27539ee5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:40.023109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:40.024024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:40.024303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:40.025180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] PATH_INFO: `/v3/projects/a0d938c674cb43faae751b86e5ce54a5/users/771d190d4a174b139499028e3a028292/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:40.052047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d938c674cb43faae751b86e5ce54a5, user_id=771d190d4a174b139499028e3a028292, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:40.054915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:40.080740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 974/1986] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:40 2022] PUT /identity/v3/projects/a0d938c674cb43faae751b86e5ce54a5/users/771d190d4a174b139499028e3a028292/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:40.085407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:40.097134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbc05-c51d-43ab-85d5-7f545b15d44a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:40.159844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:40.160523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:40.160771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:40.161068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:40.161976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:40.163893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:40.179365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1013/1987] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:40.183783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:40.194922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef18fbc-48c4-4890-9a28-430faed6beca admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:40.200170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:40.201147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:40.201426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:40.201995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] PATH_INFO: `/v3/projects/a0d938c674cb43faae751b86e5ce54a5/users/771d190d4a174b139499028e3a028292/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:40.203770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d938c674cb43faae751b86e5ce54a5, user_id=771d190d4a174b139499028e3a028292, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:40.206679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:40.218647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 975/1988] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:40 2022] PUT /identity/v3/projects/a0d938c674cb43faae751b86e5ce54a5/users/771d190d4a174b139499028e3a028292/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:40.309658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:40.311938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:40.368113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:40.388668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d58efa-053e-4093-b3dd-b10bdcf4095c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:40.496526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:40.497615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:40.497896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:40.498167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:40.500107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:40.607155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:40.608641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d09f-b7ff-4878-80f9-c2e8e93385d2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:40.624263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1014/1989] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:43.529517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:43.531824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:43.614796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:43.687922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06267b3-ff33-4b05-86a5-06f68a9991e5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:43.694630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:43.696059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:43.696517 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:43.696955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] PATH_INFO: `/v3/users/554e9190e72649f79c2451a04cc17fd7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:43.698371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] RBAC: Authorizing `identity:delete_user(user_id=554e9190e72649f79c2451a04cc17fd7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:43.700513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:43.779613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554e9190e72649f79c2451a04cc17fd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:43.792361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554e9190e72649f79c2451a04cc17fd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:43.817511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554e9190e72649f79c2451a04cc17fd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:43.818298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554e9190e72649f79c2451a04cc17fd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:43.819114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554e9190e72649f79c2451a04cc17fd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:43.840372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 976/1990] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:43 2022] DELETE /identity/v3/users/554e9190e72649f79c2451a04cc17fd7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:43.845597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:43.846125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:43.846251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:43.864187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:43.865530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:43.979555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1015/1991] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:43.991752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:43.995148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:44.136362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ee47-bcf9-4623-9914-a49e93b771db admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:44.142111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:44.142797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:44.143463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:44.143463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:44.154253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:44.155847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:44.172733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 977/1992] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:44.710418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:44.713854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:44.750311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc024b-367c-4786-a24e-5874420f2d5d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:44.755297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:44.756374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:44.756788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:44.757190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] PATH_INFO: `/v3/projects/f5bff2c6fc3b4d57aaae23a58e6cfd37` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:44.763728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5bff2c6fc3b4d57aaae23a58e6cfd37)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:44.763728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:44.893799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:44.896934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5bff2c6fc3b4d57aaae23a58e6cfd37', 'request_id': 'req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:44.909863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5bff2c6fc3b4d57aaae23a58e6cfd37', 'request_id': 'req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:44.921002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5bff2c6fc3b4d57aaae23a58e6cfd37', 'request_id': 'req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:44.951018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] The token cache is being invalidate because project f5bff2c6fc3b4d57aaae23a58e6cfd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:44.951636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:44.952532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:44.954683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1016/1993] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:44 2022] DELETE /identity/v3/projects/f5bff2c6fc3b4d57aaae23a58e6cfd37 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:44.958822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:44.977606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a810a33-b04d-4d9f-8303-a68a057070a6 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:45.052988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:45.077463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:45.078382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:45.078662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:45.078945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] PATH_INFO: `/v3/users/d1e71e70fb4e4b1c8d9775138ecbacf6` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:45.082366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e71e70fb4e4b1c8d9775138ecbacf6)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:45.084964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:45.218236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e71e70fb4e4b1c8d9775138ecbacf6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:45.231736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e71e70fb4e4b1c8d9775138ecbacf6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:45.246521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e71e70fb4e4b1c8d9775138ecbacf6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:45.249548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e71e70fb4e4b1c8d9775138ecbacf6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:45.252378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e71e70fb4e4b1c8d9775138ecbacf6'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:45.310961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 978/1994] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:44 2022] DELETE /identity/v3/users/d1e71e70fb4e4b1c8d9775138ecbacf6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:46.005104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:46.008318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:46.045447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14adb8-b036-4e1a-ba0c-1acc94a3317e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:46.157292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:46.158387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:46.158952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:46.159372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] PATH_INFO: `/v3/projects/4c581d827376466196715f750334af04` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:46.161385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c581d827376466196715f750334af04)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:46.163559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:46.325650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c581d827376466196715f750334af04', 'request_id': 'req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.346896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c581d827376466196715f750334af04', 'request_id': 'req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.361674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c581d827376466196715f750334af04', 'request_id': 'req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.386123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] The token cache is being invalidate because project 4c581d827376466196715f750334af04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:46.387281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.388156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.390060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1017/1995] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:46 2022] DELETE /identity/v3/projects/4c581d827376466196715f750334af04 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:46.394275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:46.396015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:46.421205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702604-982e-46c9-8289-07579ea37578 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:46.596398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:46.599015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:46.599576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:46.601113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] PATH_INFO: `/v3/users/673f68f7fcfb4a4ebd024e0f5594a608` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:46.615157 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] RBAC: Authorizing `identity:delete_user(user_id=673f68f7fcfb4a4ebd024e0f5594a608)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:46.623024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:46.717263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673f68f7fcfb4a4ebd024e0f5594a608'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.742446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673f68f7fcfb4a4ebd024e0f5594a608'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.759243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673f68f7fcfb4a4ebd024e0f5594a608'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.760976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673f68f7fcfb4a4ebd024e0f5594a608'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.762016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673f68f7fcfb4a4ebd024e0f5594a608'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:46.802116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:46.829077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 979/1996] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:46 2022] DELETE /identity/v3/users/673f68f7fcfb4a4ebd024e0f5594a608 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:47.520721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:47.522454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:47.542128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6d50-9239-4dd0-ab42-0e0bcbf541b2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:47.661479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:47.663367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:47.688744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:47.691131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:47.691585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:47.691860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] PATH_INFO: `/v3/projects/f69b12a6089746d4a9eec6e17af64704` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:47.693227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f69b12a6089746d4a9eec6e17af64704)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:47.696154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:47.706426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:47.789184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69b12a6089746d4a9eec6e17af64704', 'request_id': 'req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:47.803855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69b12a6089746d4a9eec6e17af64704', 'request_id': 'req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:47.815534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e6ae-4f84-4c2a-b9a9-0dc59d17249e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:47.821588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69b12a6089746d4a9eec6e17af64704', 'request_id': 'req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:47.851669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] The token cache is being invalidate because project f69b12a6089746d4a9eec6e17af64704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:47.851968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:47.854170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-389e0eb9-fec6-412f-b9ae-ed778e7c49bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:47.857385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1018/1997] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:47 2022] DELETE /identity/v3/projects/f69b12a6089746d4a9eec6e17af64704 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:47.865409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce5e7-0e97-4418-a22d-1a25c6d350b0 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:47.866299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce5e7-0e97-4418-a22d-1a25c6d350b0 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:47.866577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce5e7-0e97-4418-a22d-1a25c6d350b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:47.899408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e88ce5e7-0e97-4418-a22d-1a25c6d350b0 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:47.901798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e88ce5e7-0e97-4418-a22d-1a25c6d350b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:47.948347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:47.949290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:47.949578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:47.949852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] PATH_INFO: `/v3/users/37f130c078f0411fbf2f1ed952b7b640` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:47.952717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f130c078f0411fbf2f1ed952b7b640)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:47.954746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.023265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1019/1998] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:48.032838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437968-cef2-4818-b409-07f5915310fa None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.033483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437968-cef2-4818-b409-07f5915310fa None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.033833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437968-cef2-4818-b409-07f5915310fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.046629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f130c078f0411fbf2f1ed952b7b640'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:48.087979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5e437968-cef2-4818-b409-07f5915310fa None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:48.090841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f130c078f0411fbf2f1ed952b7b640'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:48.093267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5e437968-cef2-4818-b409-07f5915310fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.106188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1020/1999] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:48.111958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e437968-cef2-4818-b409-07f5915310fa None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.120527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f130c078f0411fbf2f1ed952b7b640'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:48.121397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f130c078f0411fbf2f1ed952b7b640'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:48.123129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2adfd779-2fcc-42cf-895b-b58b052830b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f130c078f0411fbf2f1ed952b7b640'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:48.129056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e437968-cef2-4818-b409-07f5915310fa None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.142982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.145108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.146780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.147139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.147984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:48.148805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.150287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 980/2000] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:05:47 2022] DELETE /identity/v3/users/37f130c078f0411fbf2f1ed952b7b640 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:48.153538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.154867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.155254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.155626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.177459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1021/2001] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:48.182946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.185078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.190653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:48.192964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.204709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db2bb5-37c3-42ec-bc50-d0e3092ed414 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.215923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:48.283460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 981/2002] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:48.292630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.293605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.293987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.294370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.297705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.297990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.299306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.300408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1bbb9-9bdb-4841-890b-1cdac6b53b8d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.328592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:48.338571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1022/2003] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:48.343602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.343779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.343931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.376529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] MFA Rules not processed for user `771d190d4a174b139499028e3a028292`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:48.377576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8e6f5-efc9-4370-a9fb-268f0a545476 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.377803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.435929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.437479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.438029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.438552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.477222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1023/2004] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:48.482074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.553234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.555055 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.567699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 982/2005] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:48.568723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.570263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.587521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e23c45-8bfe-40ed-baf3-3fcbf9b910f8 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.595421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.596624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.597106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.597527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.598400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.600681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.604184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:48.627567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1024/2006] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:48.631847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.642162 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4979333-84e9-43aa-aabb-3b35c40725bf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.647332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.648010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.648291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.648566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.649358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.651127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.665095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1025/2007] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:48.665308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59857dc2-ede3-4933-bc85-ad6af75a6d5d service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.666131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.672172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.672944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.674140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.674140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.676752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.677355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c82882-1779-4c63-ba98-0ec5771838b2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.681994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.682669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.682950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.683222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.683948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.685614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.703340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1026/2008] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:48.705085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.716236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b3dff-a9fd-4f57-b2d9-18cde8cfeb0b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:48.721873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:48.722915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.723285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.723660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] PATH_INFO: `/v3/projects/ebb13267683c4b1993875fb50b151dc7/users/147a9e21c6784c7586cf422852646e2b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.742646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb13267683c4b1993875fb50b151dc7, user_id=147a9e21c6784c7586cf422852646e2b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.746789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.755394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:48.757261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb965b-55a3-4dea-9499-fe2715140635 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:48.770068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 983/2009] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:48.781135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1027/2010] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:48 2022] PUT /identity/v3/projects/ebb13267683c4b1993875fb50b151dc7/users/147a9e21c6784c7586cf422852646e2b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:48.792494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:48.792779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:48.793087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:48.832244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] MFA Rules not processed for user `147a9e21c6784c7586cf422852646e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:48.834303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:48.927019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 984/2011] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:48.931416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.946946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:48.950365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.023142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:49.059041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:49.083489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b35dc1-4255-4f60-9570-b06c930204b5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.087828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fa5dc-da80-48e1-a114-9b3abe36df8e None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.090838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.092015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.092527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.092842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.093115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.093906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.094090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.094380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.094654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] PATH_INFO: `/v3/projects/9f9810b5c4034f27a3c82428fa95c593` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.096347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.101887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9810b5c4034f27a3c82428fa95c593)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.104007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.132959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1028/2012] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:48 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:49.136166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.161183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9810b5c4034f27a3c82428fa95c593', 'request_id': 'req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:49.168544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9810b5c4034f27a3c82428fa95c593', 'request_id': 'req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:49.177607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9810b5c4034f27a3c82428fa95c593', 'request_id': 'req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:49.215985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] The token cache is being invalidate because project 9f9810b5c4034f27a3c82428fa95c593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:05:49.216323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:49.217646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-eeb899d0-f617-41bf-99eb-d5f3ccb3f52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:05:49.220859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 985/2013] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:05:48 2022] DELETE /identity/v3/projects/9f9810b5c4034f27a3c82428fa95c593 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:05:49.226553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0fc5b-3993-4da4-931e-3f5bfb1386e6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.226924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0fc5b-3993-4da4-931e-3f5bfb1386e6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.227274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0fc5b-3993-4da4-931e-3f5bfb1386e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.233309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792ce94-5dbc-4956-ad83-4bca22543a45 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.284537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-72e0fc5b-3993-4da4-931e-3f5bfb1386e6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:49.286406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-72e0fc5b-3993-4da4-931e-3f5bfb1386e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.363026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.364123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.365028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.365142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.366538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.370208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.394765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 986/2014] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:49.397082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.397417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.398206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.405737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1029/2015] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:49.410005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.411772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.432766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718f509-d854-46b0-b25f-614cb9451e13 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.438891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.439713 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.440010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.441654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.442413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.445050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.453475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:49.455831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.458607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1030/2016] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:49.462671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.473907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 987/2017] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:05:49.475668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.478784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c269486-4788-49d0-a85a-4822f2967bdd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.484573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.485525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.485862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.486301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] PATH_INFO: `/v3/projects/2ba1eb9e208748ea9cce5606ade61acc/users/f15739ba00c249c3bb23c8fa1addfcfd/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.490991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf41a2-1290-4f7d-bd3f-4432d53b7826 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.496377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.497882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.498998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.499959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.501113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:05:49.501823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.506665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.507253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba1eb9e208748ea9cce5606ade61acc, user_id=f15739ba00c249c3bb23c8fa1addfcfd, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.510326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.531573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 988/2018] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:05:49.533127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.537575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1031/2019] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:49 2022] PUT /identity/v3/projects/2ba1eb9e208748ea9cce5606ade61acc/users/f15739ba00c249c3bb23c8fa1addfcfd/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:49.538460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.545980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a606b-d281-4719-9cbc-e7f948a46020 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.574327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:49.633720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b4649-952c-4fcb-8b3a-96b5efe043d8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.639225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.640143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.640496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.640843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.641714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.643549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.656647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1032/2020] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:49.665423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.675432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.676213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.676501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.676820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.677724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.683799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b98b6-33df-4c25-a21e-fc4f38b50d63 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.690690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.692076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.692813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.693226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.696277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] PATH_INFO: `/v3/projects/2ba1eb9e208748ea9cce5606ade61acc/users/f15739ba00c249c3bb23c8fa1addfcfd/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.696807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba1eb9e208748ea9cce5606ade61acc, user_id=f15739ba00c249c3bb23c8fa1addfcfd, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.699835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6fc2f-37d3-4fdb-97b4-175973914b56 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.701773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:49.724846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1033/2021] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:49 2022] PUT /identity/v3/projects/2ba1eb9e208748ea9cce5606ade61acc/users/f15739ba00c249c3bb23c8fa1addfcfd/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:49.727050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.727342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.727726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.756534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 989/2022] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:49.760140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.772426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] MFA Rules not processed for user `f15739ba00c249c3bb23c8fa1addfcfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:49.772426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.873918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1034/2023] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:49.882250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae6a10-7baa-4db1-a79f-dc23f2cdab1a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.886825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.888282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:49.888568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.889331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.889726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.890003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.891863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.894450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.925836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 990/2024] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:05:49 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:05:49.927307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.943038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf220bf-73e3-45bf-8cc9-d9737a356491 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.949574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:49.950494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:49.950953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:49.951361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:49.952236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:49.954558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:49.972383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 991/2025] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:05:49.974774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:49.998369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cc94-5610-48b4-aeea-299fefe6688b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:49.999801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2150c81-e889-4529-ad2d-58b022ea3de7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:50.004805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:50.005710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:50.006786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:50.007161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:50.008207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:50.010507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:50.016400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:50.017716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:50.018182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:50.019158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:50.021049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:50.039625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 992/2026] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:05:50.040770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:50.054243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e81e2d-099e-42ce-a4e8-b48ca28e1586 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:50.061022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:05:50.064863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:50.065857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:50.066732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:50.066732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] PATH_INFO: `/v3/projects/c3c7615479d34380a3156e615577695e/users/5aa77bf90c1045f98641dada20610fd7/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:50.099336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c7615479d34380a3156e615577695e, user_id=5aa77bf90c1045f98641dada20610fd7, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:50.102646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c38425-547c-4cab-b09e-d81986f78424 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:50.124975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 993/2027] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:05:50 2022] PUT /identity/v3/projects/c3c7615479d34380a3156e615577695e/users/5aa77bf90c1045f98641dada20610fd7/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:05:50.129394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:50.137076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:50.141742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:50.143491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:50.150991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:50.177158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1035/2028] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:05:50.219638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] MFA Rules not processed for user `5aa77bf90c1045f98641dada20610fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:05:50.220886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:05:50.312134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 994/2029] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:05:50.321385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:05:50.340558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88161f40-1c3a-4b66-ba5e-a3302105481a service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:05:50.449677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:05:50.450357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:05:50.450642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:05:50.450922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:05:50.471076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:05:50.472763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:05:50.484115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1036/2030] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:00.304124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:00.478795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=36 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:01.414687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:01.417068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:01.432103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b608f-d21f-4f2b-8d92-16aed5e086bf None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:01.436032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:01.436786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:01.437238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:01.437529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:01.504214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:01.505730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f8f8-10d1-4010-aaba-73596472a3b7 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:01.516346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 995/2031] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:03.747109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:03.817441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16900d-e649-4557-a8d7-e907b5e41888 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:03.822238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:03.823131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:03.823412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:03.823686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] PATH_INFO: `/v3/users/5aa77bf90c1045f98641dada20610fd7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:03.824947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa77bf90c1045f98641dada20610fd7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:03.826882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:03.877379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa77bf90c1045f98641dada20610fd7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:03.885171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa77bf90c1045f98641dada20610fd7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:03.892939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa77bf90c1045f98641dada20610fd7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:03.893795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa77bf90c1045f98641dada20610fd7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:03.894523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa77bf90c1045f98641dada20610fd7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:03.914708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1037/2032] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:03 2022] DELETE /identity/v3/users/5aa77bf90c1045f98641dada20610fd7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:03.919506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:03.919999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:03.920088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:03.938550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:03.939769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:04.000758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 996/2033] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:04.014467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.015990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:04.095194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2a9cc-a09d-4d8b-8979-448ac4627f91 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.098926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.099589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.099872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.100145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.110378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.111830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34a0c3-e1bc-4797-9041-92a9fc99eca2 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:04.122839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1038/2034] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:04.411125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.413018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:04.428082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4563-cd72-4841-b0ad-656b35c33b12 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.433406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.434371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.434662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.434906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] PATH_INFO: `/v3/projects/c3c7615479d34380a3156e615577695e` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.436187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c7615479d34380a3156e615577695e)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.438906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:04.486840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c7615479d34380a3156e615577695e', 'request_id': 'req-8d2558bd-4dab-400e-931e-3e83a7353637'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:04.493234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c7615479d34380a3156e615577695e', 'request_id': 'req-8d2558bd-4dab-400e-931e-3e83a7353637'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:04.499569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c7615479d34380a3156e615577695e', 'request_id': 'req-8d2558bd-4dab-400e-931e-3e83a7353637'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:04.513448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] The token cache is being invalidate because project c3c7615479d34380a3156e615577695e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:04.513731 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:04.514355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8d2558bd-4dab-400e-931e-3e83a7353637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:04.515948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 997/2035] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:04 2022] DELETE /identity/v3/projects/c3c7615479d34380a3156e615577695e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:04.523198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.523485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.523760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.543226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:04.544445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:04.562375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:04.612247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1039/2036] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:04.620799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.621117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.622095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.645836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:04.647687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:04.661044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 998/2037] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:04.663907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.674656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b6975-32c1-4528-8412-bfee945b75f1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.683101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.683972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.684101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.684309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.685330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:04.685875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.687281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:04.708559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1040/2038] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:06:04.709946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.720327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2583967-d1db-44fe-bf10-a1fc023f27d6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.725712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.726380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.726659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.727058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.727773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.729822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:04.742085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:04.778019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 999/2039] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:04.779258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.799753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6611571-f799-49f7-a877-aca280d2c7d7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.917381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.918181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.918532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.918865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.919579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.921715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:04.949957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1041/2040] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:04 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:04.954125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:04.967363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58820cfc-77df-459d-a610-a257c15b2976 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:04.975206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:04.975970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:04.976327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:04.976691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:04.978058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:04.979788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.005551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1000/2041] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:05.007373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.021850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06176150-a0b3-41e9-8a31-1665f7ac3741 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.035346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.036527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.036819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.037129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.041915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.044792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588c754-ba61-47e8-9c09-2b214dfe5fe1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.066992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1042/2042] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:05.068212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.081185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340cee0-6a31-4959-9cd6-eb366cd007dd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.086422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.087513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.087954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.088384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] PATH_INFO: `/v3/projects/3c6b29126db64fb6b43f6cf409cec45f/users/87f2c524419f47cd88652acd6d054a55/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.112010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6b29126db64fb6b43f6cf409cec45f, user_id=87f2c524419f47cd88652acd6d054a55, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.114774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.135532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1001/2043] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:05 2022] PUT /identity/v3/projects/3c6b29126db64fb6b43f6cf409cec45f/users/87f2c524419f47cd88652acd6d054a55/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:05.146742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.147027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.147300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.176654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] MFA Rules not processed for user `87f2c524419f47cd88652acd6d054a55`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:05.177944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:05.257263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1043/2044] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:05.261849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.354885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2134a46-4b83-4e21-8a89-c12b7e5df51d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.360963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.361688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.362025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.362351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.369488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.371528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.400372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1002/2045] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:05.404356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.415500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aebe97-b47e-47b1-956f-c334173898ed None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.427787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:05.478422 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.479106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.479386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.479674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.480363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.482415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.506125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1044/2046] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:05.510243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.523012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95387b-a2e4-404c-81b7-0427fab37b79 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.527919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.528578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.528874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.529195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.529928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.531553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.547106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1003/2047] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:05.549039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.560083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b251-bbae-499f-a138-06944c651954 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.565367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.566036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.566318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.566591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.567320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.568989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e3f2a-9160-4bd3-95f7-9fa70ef10eb9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.583592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1045/2048] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:05.585335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.597035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790f283-069e-4ce9-8608-adecc7a6decd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.601996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.603048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.603336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.603607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] PATH_INFO: `/v3/projects/d93b99d799b34c5b9ce34526bbf38b31/users/7a6b169d5c8046549ae8b273408fa8a8/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.625500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93b99d799b34c5b9ce34526bbf38b31, user_id=7a6b169d5c8046549ae8b273408fa8a8, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.628180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.643469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1004/2049] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:05 2022] PUT /identity/v3/projects/d93b99d799b34c5b9ce34526bbf38b31/users/7a6b169d5c8046549ae8b273408fa8a8/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:05.654975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.655194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.655475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.685414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] MFA Rules not processed for user `7a6b169d5c8046549ae8b273408fa8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:05.686672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:05.758921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1046/2050] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:05.761031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.835083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01525-5f8b-4883-b9eb-654c56d62433 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.839866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.840615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.840892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.841542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.842224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.844283 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.852894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:05.874916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1005/2051] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:05.876429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:05.888952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f7f2c-c66a-4506-9cb4-a95e74cde529 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:05.962737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:05.963446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:05.963781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:05.964062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:05.964775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:05.967164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:05.993647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1047/2052] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:05 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:05.997719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:06.009777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9305ccf-c17e-453b-a546-312565433989 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:06.015371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:06.016184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.016524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.017067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.017492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:06.019134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:06.032856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1006/2053] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:06.037801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:06.048863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ac5c1-4f69-44df-94a0-1c133d938b2b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:06.055643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:06.056585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.056870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.057185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] PATH_INFO: `/v3/projects/62c31e3e01604d668f9915717951a789/users/79847b530b4a480e83e91109805afd7d/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.081438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:06.086275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c31e3e01604d668f9915717951a789, user_id=79847b530b4a480e83e91109805afd7d, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:06.089169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:06.108577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1048/2054] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:06 2022] PUT /identity/v3/projects/62c31e3e01604d668f9915717951a789/users/79847b530b4a480e83e91109805afd7d/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:06.112835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:06.123858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f755b4-3159-4512-a79d-e3e38ff326e4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:06.189409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:06.190211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.190568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.190906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.191924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:06.193585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfc354-fd88-4295-a536-2f0a3f370546 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:06.204087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1007/2055] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:06.208303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:06.220187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841156e-dc9c-4ec8-ad91-7d7edaf9f218 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:06.224996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:06.225885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.226160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.226428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] PATH_INFO: `/v3/projects/62c31e3e01604d668f9915717951a789/users/79847b530b4a480e83e91109805afd7d/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.228523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c31e3e01604d668f9915717951a789, user_id=79847b530b4a480e83e91109805afd7d, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:06.231217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:06.244415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1049/2056] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:06 2022] PUT /identity/v3/projects/62c31e3e01604d668f9915717951a789/users/79847b530b4a480e83e91109805afd7d/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:06.255237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e361b7-1af3-4268-86b1-52799ccc8fb2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.255530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e361b7-1af3-4268-86b1-52799ccc8fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.255811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e361b7-1af3-4268-86b1-52799ccc8fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.284546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-57e361b7-1af3-4268-86b1-52799ccc8fb2 None None] MFA Rules not processed for user `79847b530b4a480e83e91109805afd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:06.285854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-57e361b7-1af3-4268-86b1-52799ccc8fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:06.363568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1008/2057] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:06.377901 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:06.380113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:06.460010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb85bb-925f-4f6d-b839-994419cccef3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:06.464611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:06.465323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:06.465604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:06.465873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:06.533424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:06.535646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba65e45-303b-4ca0-a5fe-e10220fcacdd service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:06.545286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1050/2058] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:08.622487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:08.622979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:08.623100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:08.624534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1009/2059] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:06:08.631351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:08.632067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:08.632359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:08.651909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:08.653183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:08.687839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:08.690167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:08.724541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1051/2060] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:08.736957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:08.738513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:08.751971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f0a2d-c157-42cc-805d-856216aa60b9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:08.757441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:08.758793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:08.759152 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:08.759579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:08.762439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13222c0-c300-4aee-945b-611ae92082e2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:08.767840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:08.768726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:08.769035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:08.769308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] PATH_INFO: `/v3/users/d6275dbb33b544dcb91453a858073606` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:08.770542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6275dbb33b544dcb91453a858073606)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:08.771773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:08.772484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:08.773407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:08.774207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:08.784737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1052/2061] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:06:08.823097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6275dbb33b544dcb91453a858073606'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:08.831280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6275dbb33b544dcb91453a858073606'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:08.840516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6275dbb33b544dcb91453a858073606'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:08.841742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6275dbb33b544dcb91453a858073606'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:08.842364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5174b274-4d15-4a6a-9048-d9b86be654c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6275dbb33b544dcb91453a858073606'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:08.861578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1010/2062] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:08 2022] DELETE /identity/v3/users/d6275dbb33b544dcb91453a858073606 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:09.040721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.053844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb28cec-fa2c-4b2b-ab87-f39f279bf57d service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.067294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:09.116073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.116979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.117259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.117527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] PATH_INFO: `/v3/projects/e1632e95c1054b4e89203492e13ad3b7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.118749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1632e95c1054b4e89203492e13ad3b7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.120693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.169349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1632e95c1054b4e89203492e13ad3b7', 'request_id': 'req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:09.175871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1632e95c1054b4e89203492e13ad3b7', 'request_id': 'req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:09.181654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1632e95c1054b4e89203492e13ad3b7', 'request_id': 'req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:09.196692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] The token cache is being invalidate because project e1632e95c1054b4e89203492e13ad3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:09.197098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:09.197812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2f048ac-1787-41c2-a7b9-57e4ab2d8993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:09.199687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1053/2063] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:09 2022] DELETE /identity/v3/projects/e1632e95c1054b4e89203492e13ad3b7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:09.204569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.204884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.205199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.224632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:09.225897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:09.297296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.297594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.297915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.299423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1011/2064] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:09.317810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:09.319028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:09.327275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1054/2065] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:09.331511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.342048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f7d3b-a270-49a8-a773-af0147289b4c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.347198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.347862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.348146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.348409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.349204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:09.349488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.351109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.363030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1012/2066] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:06:09.367255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.376552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe63a-b4ae-4550-b2a6-96350b2248ce None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.381383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.382040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.382325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.382606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.383306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.385274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.412362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1055/2067] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:09 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:09.416316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.427273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14870ff-11c8-4717-a38a-cf7724387d91 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.493245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.493944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.494244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.494500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.495217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.497600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.507468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:09.526954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1013/2068] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:09 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:09.527972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.538758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220306e0-9d47-4325-b732-bc4bc2415771 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.544512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.545225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.545507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.545780 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.546519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.548700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.561057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1056/2069] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:09.565682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.575853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a53b-6d94-4e5f-85b1-1eda53bab6bd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.581569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.582260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.582545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.582824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.583697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.585502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875e91f-a0c4-4dd9-9906-35faa4dbba6f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.601646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1014/2070] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:09.603005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.613735 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac12529-cc14-47d1-9777-206ead54b6d9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.618290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.619184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.619461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.619732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] PATH_INFO: `/v3/projects/d07093b82f094ae398180a96c2d335f3/users/55dcf415863e4a948edab00c7bff5f98/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.634958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07093b82f094ae398180a96c2d335f3, user_id=55dcf415863e4a948edab00c7bff5f98, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.637942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.651862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1057/2071] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:09 2022] PUT /identity/v3/projects/d07093b82f094ae398180a96c2d335f3/users/55dcf415863e4a948edab00c7bff5f98/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:09.663160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.663621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.663710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.691833 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] MFA Rules not processed for user `55dcf415863e4a948edab00c7bff5f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:09.693085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:09.760105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1015/2072] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:09.776364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:09.778378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:09.827593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:09.858526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181b05a-8a11-4e19-88f4-f582f61386a0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:09.864772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:09.865451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:09.865730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:09.865981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:09.873912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:09.876090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:09.884841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1058/2073] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:15.557550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:15.558715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:15.624379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936fda2-af8b-4537-829d-7e84de040b85 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:15.631756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:15.632439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:15.632721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:15.633021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:15.634999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:15.695758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:15.697397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:15.707614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1016/2074] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:16.348892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:16.359154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f695b44-b2f7-4676-a7e7-8b2490013393 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:16.363229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:16.363886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:16.364176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:16.364474 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:16.366432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:16.433919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:16.435371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:16.448125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1059/2075] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:22.253402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:22.297003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:22.361597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b9d60-05c2-427d-b31c-9f96b5769303 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:22.364433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:22.365971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:22.366637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:22.368340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.368717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.369082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] PATH_INFO: `/v3/users/55dcf415863e4a948edab00c7bff5f98` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.370441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dcf415863e4a948edab00c7bff5f98)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:22.372568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:22.384564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aebb9-8ba2-45eb-93a9-9104087a9620 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:22.390000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:22.391180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.391597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.392080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] PATH_INFO: `/v3/users/147a9e21c6784c7586cf422852646e2b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.394113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147a9e21c6784c7586cf422852646e2b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:22.396540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:22.468237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dcf415863e4a948edab00c7bff5f98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.482669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dcf415863e4a948edab00c7bff5f98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.484246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147a9e21c6784c7586cf422852646e2b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.501186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147a9e21c6784c7586cf422852646e2b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.509975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dcf415863e4a948edab00c7bff5f98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.515798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147a9e21c6784c7586cf422852646e2b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.517546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147a9e21c6784c7586cf422852646e2b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.518393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dcf415863e4a948edab00c7bff5f98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.518720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d5b2eaca-a936-40ed-8d08-5501f7594124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147a9e21c6784c7586cf422852646e2b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.519137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5e6dca02-0ab2-4e68-84f5-db6c57023b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dcf415863e4a948edab00c7bff5f98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:22.553183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1017/2076] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:22 2022] DELETE /identity/v3/users/55dcf415863e4a948edab00c7bff5f98 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:22.553817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1060/2077] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:22 2022] DELETE /identity/v3/users/147a9e21c6784c7586cf422852646e2b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:22.554873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.555350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.555644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.562576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.562576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.562964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.587836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:22.589102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:22.593607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:22.595224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:22.628869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:22.666213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1061/2078] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:22.678382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1018/2079] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:22.678572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:22.678572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:22.686753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:22.688338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:22.738290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76542fd0-3031-4f90-b519-80094d0215fb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:22.742751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:22.743593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.744795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:22.745274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.745577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.752613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6679-de75-49b1-ad8f-a9d991ff3d77 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:22.757836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:22.758787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:22.759137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:22.760239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:22.760880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:22.761214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:22.772017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1062/2080] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:06:22.778422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:22.780084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:22.798443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1019/2081] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:06:23.239070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.240542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.245607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.247054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.261469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad63c-a05a-440d-a936-8250797f0d9d service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.265259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ce9d-c27e-4513-ab39-65d1688e3cef service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.271262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.272133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.272425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.272678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] PATH_INFO: `/v3/projects/ebb13267683c4b1993875fb50b151dc7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.274541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.274678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb13267683c4b1993875fb50b151dc7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.276726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.276962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.277782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.278236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] PATH_INFO: `/v3/projects/d07093b82f094ae398180a96c2d335f3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.279687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07093b82f094ae398180a96c2d335f3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.281950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.334174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb13267683c4b1993875fb50b151dc7', 'request_id': 'req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.339883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07093b82f094ae398180a96c2d335f3', 'request_id': 'req-f6210705-a86a-4d32-b2ed-811dcfcdc075'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.340752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb13267683c4b1993875fb50b151dc7', 'request_id': 'req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.347101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb13267683c4b1993875fb50b151dc7', 'request_id': 'req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.347527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07093b82f094ae398180a96c2d335f3', 'request_id': 'req-f6210705-a86a-4d32-b2ed-811dcfcdc075'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.353739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07093b82f094ae398180a96c2d335f3', 'request_id': 'req-f6210705-a86a-4d32-b2ed-811dcfcdc075'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.362663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] The token cache is being invalidate because project ebb13267683c4b1993875fb50b151dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:23.362931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.363575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.365234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1063/2082] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:23 2022] DELETE /identity/v3/projects/ebb13267683c4b1993875fb50b151dc7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:23.367382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] The token cache is being invalidate because project d07093b82f094ae398180a96c2d335f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:23.367816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.368498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-f6210705-a86a-4d32-b2ed-811dcfcdc075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.369279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.370317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1020/2083] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:23 2022] DELETE /identity/v3/projects/d07093b82f094ae398180a96c2d335f3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:23.371075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.374881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f814-dfa9-4f4a-a2a8-88478257472a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.375167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f814-dfa9-4f4a-a2a8-88478257472a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.375443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f814-dfa9-4f4a-a2a8-88478257472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.394230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cde3f814-dfa9-4f4a-a2a8-88478257472a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:23.395323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cde3f814-dfa9-4f4a-a2a8-88478257472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.435211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d7b9c-f30f-411e-9bfe-29a5eeccfb70 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.439080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:23.455759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1021/2084] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:23.464013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.464316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.464588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.484235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:23.485479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.494427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1022/2085] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:23.498468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.500094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.501024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.501310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.501583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] PATH_INFO: `/v3/users/f15739ba00c249c3bb23c8fa1addfcfd` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.502805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15739ba00c249c3bb23c8fa1addfcfd)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.504750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.507974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fc78a-6ed7-4e4d-bf0b-0f23e6251729 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.512627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.513305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.513597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.513848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.514740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:23.515076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.516644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.532838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1023/2086] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:06:23.533946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.546441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67089d-11b4-4409-ae93-307041631aa4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.552044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.552732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.553052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.553320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.554390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.556059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.581794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15739ba00c249c3bb23c8fa1addfcfd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.595136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15739ba00c249c3bb23c8fa1addfcfd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.606300 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1024/2087] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:23 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:23.613786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.621137 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15739ba00c249c3bb23c8fa1addfcfd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.625115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15739ba00c249c3bb23c8fa1addfcfd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.625115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15739ba00c249c3bb23c8fa1addfcfd'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.665177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1064/2088] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:23 2022] DELETE /identity/v3/users/f15739ba00c249c3bb23c8fa1addfcfd => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:23.666416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.669846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:23.679792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.736548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104896c-34b1-4ead-90ec-31e77ec2c46a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.801985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f28b45-f0da-42ee-ab56-99a8ec2262b0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.812054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.813978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.813978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.814766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] PATH_INFO: `/v3/users/87f2c524419f47cd88652acd6d054a55` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.816091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f2c524419f47cd88652acd6d054a55)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.818752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.863019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:23.863240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.863774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.864040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.864313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.865588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.867745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.895539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f2c524419f47cd88652acd6d054a55'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.901239 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1025/2089] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:23 2022] POST /identity/v3/users => generated 283 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:23.903155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.905605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.911899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f2c524419f47cd88652acd6d054a55'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.921067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f2c524419f47cd88652acd6d054a55'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.921642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3576b41-8fed-4617-8075-3a4b5f8e8c1c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.926684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f2c524419f47cd88652acd6d054a55'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.927793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:23.928381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8fa52827-08aa-434b-a515-5c55e59762ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f2c524419f47cd88652acd6d054a55'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:23.928522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.928766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.929222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.931672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:23.932503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:23.950342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1026/2090] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:23.955092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:23.956647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:23.971174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1065/2091] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:23 2022] DELETE /identity/v3/users/87f2c524419f47cd88652acd6d054a55 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:23.975245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c077514-a1ef-4da7-9001-77c4885bf051 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:23.977252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:23.977539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:23.977895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:23.997912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:23.999227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.099021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1066/2092] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:24.100252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.101931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.114438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f209-8884-4951-be92-c4a65f401663 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.119738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.119904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.120847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.121164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.121861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.122347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.122812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.123641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] PATH_INFO: `/v3/projects/2ba1eb9e208748ea9cce5606ade61acc` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.123796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.124986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba1eb9e208748ea9cce5606ade61acc)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.125392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.127882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.139818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1027/2093] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:24.139818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.141782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.182345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba1eb9e208748ea9cce5606ade61acc', 'request_id': 'req-ca95c815-f8ac-4abd-882f-80bc31ea552c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.188628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba1eb9e208748ea9cce5606ade61acc', 'request_id': 'req-ca95c815-f8ac-4abd-882f-80bc31ea552c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.194294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba1eb9e208748ea9cce5606ade61acc', 'request_id': 'req-ca95c815-f8ac-4abd-882f-80bc31ea552c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.203861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa91c8-a7a6-43bd-923a-0dfc30539ba4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.208161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] The token cache is being invalidate because project 2ba1eb9e208748ea9cce5606ade61acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:24.208455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.209141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.211326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1067/2094] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:24 2022] DELETE /identity/v3/projects/2ba1eb9e208748ea9cce5606ade61acc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:24.212064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.213423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.217065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:24.246846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:24.263032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.263742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.264018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.264290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.266117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.283063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95c815-f8ac-4abd-882f-80bc31ea552c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.288444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.289614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.289936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.290167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] PATH_INFO: `/v3/projects/1118eaf0b9d94db09dda81f5354c9659/users/58dd18e910a240e39c9102ff96f62c01/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.305951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1118eaf0b9d94db09dda81f5354c9659, user_id=58dd18e910a240e39c9102ff96f62c01, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.308633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6d856-a07c-4873-ac7a-f6a53319bfe1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.324244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1068/2095] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:24 2022] PUT /identity/v3/projects/1118eaf0b9d94db09dda81f5354c9659/users/58dd18e910a240e39c9102ff96f62c01/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:24.325826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49a19-c1f3-4351-8188-28f37d6d7477 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.326280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49a19-c1f3-4351-8188-28f37d6d7477 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.326698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49a19-c1f3-4351-8188-28f37d6d7477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.330349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.331809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256348d3-213d-44b4-99a2-3d0481a30efd service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.342653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1028/2096] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:24.344268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.344597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.344868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.348123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-eea49a19-c1f3-4351-8188-28f37d6d7477 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:24.349569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-eea49a19-c1f3-4351-8188-28f37d6d7477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.379031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] MFA Rules not processed for user `58dd18e910a240e39c9102ff96f62c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:24.380713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.434056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1069/2097] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:24.442460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.443049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.443500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.463470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1029/2098] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:24.466730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:24.467567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.468304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.476995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1070/2099] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:24.481301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.482659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ccc4c-1deb-47c8-9396-e3c50d7fb798 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.489964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.490871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0371c3-0588-42e4-955d-c4a4d91ac3f1 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.491262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.491806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.492507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.494004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.495993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.496652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.496776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.496975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.497254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.497996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:24.498302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.499923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.534833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1071/2100] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:06:24.535974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.538497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1030/2101] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:24.539521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.547058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41676b79-921b-43b6-b681-5f0c0dd7d0b6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.573760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:24.610958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa049b-e225-4a50-b38c-9582477628ba admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.614991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.615665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.615946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.616197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.616312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.616962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.617066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.617681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.617681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.618242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.618962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.620572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.646843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1031/2102] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:24.647701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.649011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1072/2103] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:24.649011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.651374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.666148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3720b30-4335-471a-b69d-cd04762f1e11 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.675276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:24.723845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd01f-e0af-4861-bd4e-4ebc54bc24c5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.726471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.727352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.727631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.727900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] PATH_INFO: `/v3/projects/3c6b29126db64fb6b43f6cf409cec45f` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.729124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6b29126db64fb6b43f6cf409cec45f)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.729241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.729815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.730092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.730362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.731054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.731151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.732656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.743001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1073/2104] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:24.743934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.754853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f035c82-60a4-43d0-9927-baa38b4685e7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.760023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.760744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.761093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.761374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.762100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.764115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.786891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1074/2105] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:24 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:24.788123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.790114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6b29126db64fb6b43f6cf409cec45f', 'request_id': 'req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.797115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6b29126db64fb6b43f6cf409cec45f', 'request_id': 'req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.799509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c40296-790f-4ef2-a7d1-affece2ed241 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.803569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6b29126db64fb6b43f6cf409cec45f', 'request_id': 'req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.818809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] The token cache is being invalidate because project 3c6b29126db64fb6b43f6cf409cec45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:24.819097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.819848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:24.821923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1032/2106] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:24 2022] DELETE /identity/v3/projects/3c6b29126db64fb6b43f6cf409cec45f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:24.822779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.824091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.862800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.863735 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.864015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.864289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] PATH_INFO: `/v3/projects/020d21b3593b451ba5b92e800fca021b/users/f9d00ceb38464f109591a740789cc9e5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.875732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:24.882638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020d21b3593b451ba5b92e800fca021b, user_id=f9d00ceb38464f109591a740789cc9e5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.886105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.894988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca0067-cc1d-4747-a245-0c4fa3d6f05e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.901818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1075/2107] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:24 2022] PUT /identity/v3/projects/020d21b3593b451ba5b92e800fca021b/users/f9d00ceb38464f109591a740789cc9e5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:24.904954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.906285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:24.970046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.970767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.971050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.971292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.972210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.973899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.977142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39e3e-9298-4d46-87ae-8640ba4f4b55 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:24.982138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:24.983141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:24.983458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:24.983795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] PATH_INFO: `/v3/users/7a6b169d5c8046549ae8b273408fa8a8` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:24.987645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6b169d5c8046549ae8b273408fa8a8)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:24.987810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1033/2108] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:24.987810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:24.988204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:24.988852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.001854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6800ba-e28c-4f18-bfa4-95d7a88d9351 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.006575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.007306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.007588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.007886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.008614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.010274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.012679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:25.021201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1034/2109] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:25.021983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.041052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290ce40-22af-4d36-ad76-4bbc9802d2a9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.043835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.044574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.044999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.045278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.046090 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.047728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.050717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6b169d5c8046549ae8b273408fa8a8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.058969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6b169d5c8046549ae8b273408fa8a8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.067800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1035/2110] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:25.067800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.078986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6b169d5c8046549ae8b273408fa8a8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.079358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be30b60-cf99-45eb-b2ae-1b286d7885a1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.079750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6b169d5c8046549ae8b273408fa8a8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.080397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d2700179-c557-490c-87b8-23f675fbe598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6b169d5c8046549ae8b273408fa8a8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.086474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.087522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.087802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.088075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] PATH_INFO: `/v3/projects/020d21b3593b451ba5b92e800fca021b/users/f9d00ceb38464f109591a740789cc9e5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.090257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020d21b3593b451ba5b92e800fca021b, user_id=f9d00ceb38464f109591a740789cc9e5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.093652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.108110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1036/2111] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:25 2022] PUT /identity/v3/projects/020d21b3593b451ba5b92e800fca021b/users/f9d00ceb38464f109591a740789cc9e5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:25.109025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.110642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.122432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1076/2112] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:24 2022] DELETE /identity/v3/users/7a6b169d5c8046549ae8b273408fa8a8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:25.123855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.124147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.124421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.153694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] MFA Rules not processed for user `f9d00ceb38464f109591a740789cc9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:25.155051 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.188434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731ffd-f0e6-42fb-b8e3-6a46e16a9c25 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.243579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1077/2113] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:25.252550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.254710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.275318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.276599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.277357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.277465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] PATH_INFO: `/v3/projects/49858bcc57684beebb5b30e6f0817280/users/9585c13a1941436a98f6c895626c9016/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.287241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:25.296899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=49858bcc57684beebb5b30e6f0817280, user_id=9585c13a1941436a98f6c895626c9016, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.301551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77cd72-e725-4631-a9df-e2fabfa2fbbe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.321279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1037/2114] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:25 2022] PUT /identity/v3/projects/49858bcc57684beebb5b30e6f0817280/users/9585c13a1941436a98f6c895626c9016/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:25.333668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.334198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.334932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.355468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb3a1-31d7-4107-a908-d1f41037f102 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.366433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] MFA Rules not processed for user `9585c13a1941436a98f6c895626c9016`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:25.368999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.399381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:25.412167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.413201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.413648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.414019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.479541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1038/2115] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:25.485940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.490055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.491555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.501967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1078/2116] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:25.577214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.578876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.605545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ece4e-7b89-4180-84b5-cd983c0d1e0f None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.613676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.614810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.615308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.615798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.617204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.620173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.648481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1039/2117] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:25.649350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.654904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec723949-5bb8-46ce-894f-2b6d94348f84 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.705657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786521a-d94f-4295-841b-e4519175e493 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.709438 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.710094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.710766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.711047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.713211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.714637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.715524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.716208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.716483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] PATH_INFO: `/v3/projects/d93b99d799b34c5b9ce34526bbf38b31` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.717687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93b99d799b34c5b9ce34526bbf38b31)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.719681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.728213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:25.762817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:25.771367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93b99d799b34c5b9ce34526bbf38b31', 'request_id': 'req-4e869e29-f31e-4b52-bda1-c19c58cafbad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.778496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.779083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93b99d799b34c5b9ce34526bbf38b31', 'request_id': 'req-4e869e29-f31e-4b52-bda1-c19c58cafbad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.779966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.785911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93b99d799b34c5b9ce34526bbf38b31', 'request_id': 'req-4e869e29-f31e-4b52-bda1-c19c58cafbad'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.790285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1040/2118] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:25.791512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.801527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] The token cache is being invalidate because project d93b99d799b34c5b9ce34526bbf38b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:25.801819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.802309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73ddef-3e75-486b-a553-5f6653c1abf7 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.802485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.804160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1079/2119] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:25 2022] DELETE /identity/v3/projects/d93b99d799b34c5b9ce34526bbf38b31 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:25.809990 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.811563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.879418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.880336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.880547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.880815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.881553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.884191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.884618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e869e29-f31e-4b52-bda1-c19c58cafbad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.889358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.890558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.890558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.890791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] PATH_INFO: `/v3/users/79847b530b4a480e83e91109805afd7d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.892034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79847b530b4a480e83e91109805afd7d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.894103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.906538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1041/2120] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:25 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:25.914035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.917886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:25.940461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d51b13-50e6-41e5-aa76-b38945df19c7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.946918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.948183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.948615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.948965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.949746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.951604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.964252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1042/2121] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:25.968580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:25.969394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79847b530b4a480e83e91109805afd7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.978326 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79847b530b4a480e83e91109805afd7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.979821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dc44-c559-4390-a8d9-0ef4bdd6b4d3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:25.984953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:25.985364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79847b530b4a480e83e91109805afd7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.985685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:25.985899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:25.986001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79847b530b4a480e83e91109805afd7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.986170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:25.986652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-66052eff-df9a-4215-be58-f96e7ae0a9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79847b530b4a480e83e91109805afd7d'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:25.986891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:25.988514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:25.999200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1043/2122] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:26.003148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.004855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.007531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1080/2123] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:25 2022] DELETE /identity/v3/users/79847b530b4a480e83e91109805afd7d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:26.018019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64190-5ab9-4fb4-a62e-435312a68585 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.083708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.084619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.084955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.085210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] PATH_INFO: `/v3/projects/1f296f2a99ff431b842ca1a44441e4a2/users/cfe68a1a58cf4cbe87e556620c2cde30/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.104167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f296f2a99ff431b842ca1a44441e4a2, user_id=cfe68a1a58cf4cbe87e556620c2cde30, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.106973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.123095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1044/2124] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:26 2022] PUT /identity/v3/projects/1f296f2a99ff431b842ca1a44441e4a2/users/cfe68a1a58cf4cbe87e556620c2cde30/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:26.133699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.134394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.134799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.170207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] MFA Rules not processed for user `cfe68a1a58cf4cbe87e556620c2cde30`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:26.171722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.218938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:26.259124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1081/2125] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:26.263965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.272472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:26.328282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.330018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.346238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36fcd5-d14d-4cca-ad7c-ee3dfe98d730 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.352030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.353028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.353311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.353581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.354327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.356253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.388618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1045/2126] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:26.389801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.407327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bb3f9-e0c6-4a84-822f-215c3ea7a76c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.453672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcb561-b67f-488e-a6a0-1bb690c43ac6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.458413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.459105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.459380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.459653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.460396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.462378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.464124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.465037 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.465330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.465586 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] PATH_INFO: `/v3/projects/62c31e3e01604d668f9915717951a789` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.467160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c31e3e01604d668f9915717951a789)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.469104 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.484349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1046/2127] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:26.488322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.500695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734af2-0ed2-4932-b29c-496363b860de admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.505426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.506123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.506416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.506678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.507493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.509123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.520416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1047/2128] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:26.522882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c31e3e01604d668f9915717951a789', 'request_id': 'req-bbf5320e-1860-45bd-a4af-489063206fd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:26.525305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.535857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bd7e2-d9f4-4261-87ff-d33201a829f5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.548551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c31e3e01604d668f9915717951a789', 'request_id': 'req-bbf5320e-1860-45bd-a4af-489063206fd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:26.555488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c31e3e01604d668f9915717951a789', 'request_id': 'req-bbf5320e-1860-45bd-a4af-489063206fd0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:26.569262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] The token cache is being invalidate because project 62c31e3e01604d668f9915717951a789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:26.569542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:26.570332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-bbf5320e-1860-45bd-a4af-489063206fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:26.572117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1082/2129] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:26 2022] DELETE /identity/v3/projects/62c31e3e01604d668f9915717951a789 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:26.577133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc830-e195-4066-af94-118579573f09 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.577405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc830-e195-4066-af94-118579573f09 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.577679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc830-e195-4066-af94-118579573f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.597521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-5e4bc830-e195-4066-af94-118579573f09 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:26.598739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5e4bc830-e195-4066-af94-118579573f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.601175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.602016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.602296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.602575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] PATH_INFO: `/v3/projects/653ac482514b4668860268ef8a0a0701/users/73812f8963ea484e9b571f42fd395f7c/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.606762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1083/2130] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:26.615242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.615533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.615933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.618588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=653ac482514b4668860268ef8a0a0701, user_id=73812f8963ea484e9b571f42fd395f7c, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.621361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.624537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:26.629295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:26.636197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:26.637242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1048/2131] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:26 2022] PUT /identity/v3/projects/653ac482514b4668860268ef8a0a0701/users/73812f8963ea484e9b571f42fd395f7c/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:26.637841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.641529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.643196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.698922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1084/2132] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:26.703180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.707753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba98900-3afb-471e-8466-d35b2af99b90 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.712343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4578c-3543-43d2-88fb-3b99f9fe80c6 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.714155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.714827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.715091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.715359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.716449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.717438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.718110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.718207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.718388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.718665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.719391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:26.719692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.721330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.731898 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1049/2133] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:26.733073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1085/2134] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:06:26.733220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.737268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.743967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b396be-e406-4a2c-bef3-2c916f94757d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.748399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178d289-6b4b-4a37-8efc-03cb3f9726b6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.748855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.749752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.750032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.750401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] PATH_INFO: `/v3/projects/653ac482514b4668860268ef8a0a0701/users/73812f8963ea484e9b571f42fd395f7c/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.752230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=653ac482514b4668860268ef8a0a0701, user_id=73812f8963ea484e9b571f42fd395f7c, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.753554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.754251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.754539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.754812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.754905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d4ccd-efcd-4718-8a0b-e3c005bfe506 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.755503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.757708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.765647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1050/2135] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:26 2022] PUT /identity/v3/projects/653ac482514b4668860268ef8a0a0701/users/73812f8963ea484e9b571f42fd395f7c/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:26.776391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.776686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.776989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.787323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1086/2136] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:26.791343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.805841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] MFA Rules not processed for user `73812f8963ea484e9b571f42fd395f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:26.807179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.851837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb5c96-88be-4d81-9e17-c910314b1572 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.856472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.857180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.857464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.857740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.858435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.860386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.878478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1051/2137] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:26.883602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1087/2138] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:26 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:26.885106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.890211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.891916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.894933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e910c0-4821-4e07-88e3-8c0e3a12ead9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.899823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.900490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.900777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.901293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.901887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.903502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.913820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1052/2139] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:26.917590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.926881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2e061-5df3-4230-a323-3bc3c7398706 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.931775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.932417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.932695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.933005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.934279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.935656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:26.946089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1053/2140] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:26.950168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:26.950487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb5046-d8ae-4e0a-afb6-8c64114f41b9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.954445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.955099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.955377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.955651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.957477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:26.959877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb33dc-745f-4222-b67a-384c9a1469ce admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:26.964941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:26.966187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:26.966467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:26.966739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] PATH_INFO: `/v3/projects/f6a3864816c4415eba62221ed835d240/users/0f4b1e3cf1e14dc8a4aa19ea35691c1b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:26.974462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:26.984200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a3864816c4415eba62221ed835d240, user_id=0f4b1e3cf1e14dc8a4aa19ea35691c1b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:26.988056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab0500-0b2c-4b46-8848-23961df1aa8d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:27.004718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1054/2141] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:26 2022] PUT /identity/v3/projects/f6a3864816c4415eba62221ed835d240/users/0f4b1e3cf1e14dc8a4aa19ea35691c1b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:27.016106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.016447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.016741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.027789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:27.029209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:27.038251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1088/2142] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:06:27.044278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] MFA Rules not processed for user `0f4b1e3cf1e14dc8a4aa19ea35691c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:27.045525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:27.049824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:27.114891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1055/2143] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:27.126235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:27.127628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:27.138934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0640-329e-46e3-91d5-ba2cc9240913 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:27.189885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:27.190649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.192411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.192411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.202202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:27.203861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d7c0-cd21-4d04-bf5a-a1f2adb22b8b service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:27.213052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1089/2144] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:27.238801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:27.356627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73864a72-aa9d-4d2f-afc4-c7b4f51f4b38 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:27.361883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:27.363252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.371790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.371790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] PATH_INFO: `/v3/users/0f4b1e3cf1e14dc8a4aa19ea35691c1b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.371790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4b1e3cf1e14dc8a4aa19ea35691c1b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:27.371790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:27.428631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4b1e3cf1e14dc8a4aa19ea35691c1b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:27.436007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4b1e3cf1e14dc8a4aa19ea35691c1b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:27.443849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4b1e3cf1e14dc8a4aa19ea35691c1b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:27.445027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4b1e3cf1e14dc8a4aa19ea35691c1b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:27.445748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4b1e3cf1e14dc8a4aa19ea35691c1b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:27.464167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1056/2145] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:27 2022] DELETE /identity/v3/users/0f4b1e3cf1e14dc8a4aa19ea35691c1b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:27.469192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.469487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.469762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.486851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:27.488087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:27.551177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1090/2146] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:27.560778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:27.562359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:27.597290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:27.631214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b184-837a-434a-b474-04d58cf3a391 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:27.635738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:27.636389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.636656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.636929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.646192 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:27.647621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:27.657326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1057/2147] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:27.956363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:27.958029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:27.972318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b2122-334d-489d-ad33-e923c7b54fbb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:27.977972 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:27.978921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:27.979206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:27.979486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] PATH_INFO: `/v3/projects/f6a3864816c4415eba62221ed835d240` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:27.980678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a3864816c4415eba62221ed835d240)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:27.982668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:28.037455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a3864816c4415eba62221ed835d240', 'request_id': 'req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:28.044820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a3864816c4415eba62221ed835d240', 'request_id': 'req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:28.046302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:28.053542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a3864816c4415eba62221ed835d240', 'request_id': 'req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:28.067999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] The token cache is being invalidate because project f6a3864816c4415eba62221ed835d240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:28.068283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:28.068882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:28.070632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1091/2148] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:27 2022] DELETE /identity/v3/projects/f6a3864816c4415eba62221ed835d240 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:31.729768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:31.731162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:31.793507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffa17f-0b2a-4cfd-8c99-a12d6b89e459 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:31.848798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:31.849491 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:31.849767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:31.850037 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:31.852053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:31.911053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:31.912544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79f6f-1f4f-4941-a670-a817ff54a18d service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:31.922061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1058/2149] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:32.497737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:32.499414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:32.512828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8547c5-1c12-4aef-9c61-a7bdf5f28fe1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:32.517483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:32.518332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:32.518648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:32.518977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:32.520732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:32.591106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:32.594171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:32.615570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1092/2150] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:36.272018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc3e9-b23d-4660-bc30-5bc80855c0e8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:36.272584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc3e9-b23d-4660-bc30-5bc80855c0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:36.272584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afc3e9-b23d-4660-bc30-5bc80855c0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:36.291138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e9afc3e9-b23d-4660-bc30-5bc80855c0e8 None None] MFA Rules not processed for user `9f56095d6bac4f108a73790ac9004f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:36.292388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e9afc3e9-b23d-4660-bc30-5bc80855c0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:36.338165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:36.366814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1059/2151] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:40.861005 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:40.862518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:40.951252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d031cd-925b-4bc1-9a4f-eccb033ef9ac service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:40.956433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:40.957393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:40.957676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:40.957948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] PATH_INFO: `/v3/users/9585c13a1941436a98f6c895626c9016` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:40.959170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9585c13a1941436a98f6c895626c9016)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:40.961145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:41.011541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9585c13a1941436a98f6c895626c9016'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.019156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9585c13a1941436a98f6c895626c9016'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.028090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9585c13a1941436a98f6c895626c9016'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.028716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9585c13a1941436a98f6c895626c9016'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.029377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9585c13a1941436a98f6c895626c9016'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.040572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:41.048608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1093/2152] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:40 2022] DELETE /identity/v3/users/9585c13a1941436a98f6c895626c9016 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:41.053210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:41.053698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:41.053800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:41.071455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:41.073435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:41.134030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1060/2153] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:41.143947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:41.146678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:41.209960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbf461-8359-4d03-894d-dfa0cd450e53 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:41.213920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:41.214581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:41.214863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:41.215138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:41.225286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:41.226354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:41.236573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1094/2154] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:41.526624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:41.528633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:41.544058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55b6a-bae1-4659-91a8-d9aa4382eb37 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:41.557243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:41.557870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:41.558154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:41.558422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] PATH_INFO: `/v3/projects/49858bcc57684beebb5b30e6f0817280` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:41.560278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=49858bcc57684beebb5b30e6f0817280)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:41.562269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:41.657050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49858bcc57684beebb5b30e6f0817280', 'request_id': 'req-2b465855-28d4-41c6-97a0-7398cc63c59f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.663412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49858bcc57684beebb5b30e6f0817280', 'request_id': 'req-2b465855-28d4-41c6-97a0-7398cc63c59f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.669798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49858bcc57684beebb5b30e6f0817280', 'request_id': 'req-2b465855-28d4-41c6-97a0-7398cc63c59f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.687662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] The token cache is being invalidate because project 49858bcc57684beebb5b30e6f0817280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:41.687974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.688785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.694062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1061/2155] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:41 2022] DELETE /identity/v3/projects/49858bcc57684beebb5b30e6f0817280 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:41.695007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:41.717449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2030-1d51-4f28-a37c-e9fdafb99424 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:41.837448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:41.840317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:41.840317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:41.840317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] PATH_INFO: `/v3/users/cfe68a1a58cf4cbe87e556620c2cde30` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:41.841414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe68a1a58cf4cbe87e556620c2cde30)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:41.843381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:41.906949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe68a1a58cf4cbe87e556620c2cde30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.915906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe68a1a58cf4cbe87e556620c2cde30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.923424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe68a1a58cf4cbe87e556620c2cde30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.924074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe68a1a58cf4cbe87e556620c2cde30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.924667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe68a1a58cf4cbe87e556620c2cde30'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:41.943198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1095/2156] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:41 2022] DELETE /identity/v3/users/cfe68a1a58cf4cbe87e556620c2cde30 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:42.232400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:42.234543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:42.260666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465855-28d4-41c6-97a0-7398cc63c59f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:42.335290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:42.336386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:42.336667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:42.336963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] PATH_INFO: `/v3/projects/1f296f2a99ff431b842ca1a44441e4a2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:42.338890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f296f2a99ff431b842ca1a44441e4a2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:42.341194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:42.351676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:42.404577 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f296f2a99ff431b842ca1a44441e4a2', 'request_id': 'req-7bcfe0c1-db70-4780-bfe9-78d58b94f118'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.411232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f296f2a99ff431b842ca1a44441e4a2', 'request_id': 'req-7bcfe0c1-db70-4780-bfe9-78d58b94f118'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.430753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f296f2a99ff431b842ca1a44441e4a2', 'request_id': 'req-7bcfe0c1-db70-4780-bfe9-78d58b94f118'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.446645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] The token cache is being invalidate because project 1f296f2a99ff431b842ca1a44441e4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:42.447734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.448355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.450012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1062/2157] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:42 2022] DELETE /identity/v3/projects/1f296f2a99ff431b842ca1a44441e4a2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:42.454087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:42.455486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:42.474677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf9dd4-c71c-4dd0-bd7b-0bf0b8cb9f9e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:42.499732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:42.530638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:42.532506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:42.589922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:42.591746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:42.592035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:42.592303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] PATH_INFO: `/v3/users/73812f8963ea484e9b571f42fd395f7c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:42.594243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73812f8963ea484e9b571f42fd395f7c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:42.597078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:42.661242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfe0c1-db70-4780-bfe9-78d58b94f118 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:42.667648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:42.668560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:42.668992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:42.669275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] PATH_INFO: `/v3/users/56fd4fa666e149948a381671063723fa` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:42.670546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=56fd4fa666e149948a381671063723fa)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:42.673293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:42.714593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73812f8963ea484e9b571f42fd395f7c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.727803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73812f8963ea484e9b571f42fd395f7c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.737981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73812f8963ea484e9b571f42fd395f7c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.738753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73812f8963ea484e9b571f42fd395f7c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.739404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa98c551-d85e-4e4a-9b3e-f480394a5082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73812f8963ea484e9b571f42fd395f7c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.741116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56fd4fa666e149948a381671063723fa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.760792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56fd4fa666e149948a381671063723fa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.777396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56fd4fa666e149948a381671063723fa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.778297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56fd4fa666e149948a381671063723fa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.779349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56fd4fa666e149948a381671063723fa'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:42.787663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1096/2158] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:42 2022] DELETE /identity/v3/users/73812f8963ea484e9b571f42fd395f7c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:42.807010 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1063/2159] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:42 2022] DELETE /identity/v3/users/56fd4fa666e149948a381671063723fa => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:42.811883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:42.812196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:42.812568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:42.837442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:42.839110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:42.922971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1097/2160] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:42.961187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:42.963715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:43.014228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:43.096999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8783f6-8b6a-4576-9e83-047c94086b6a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.101870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.103198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.104067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.104067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.147844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.149736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e924bb-3e16-4a88-a9aa-2c48567285b2 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.162077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1064/2161] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:43.216424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.219205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:43.243947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e373-71e1-4e7b-ac42-5fc88284db80 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.255604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.256842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.257313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.257762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] PATH_INFO: `/v3/projects/653ac482514b4668860268ef8a0a0701` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.258800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653ac482514b4668860268ef8a0a0701)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.260845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.298799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:43.317199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653ac482514b4668860268ef8a0a0701', 'request_id': 'req-1061c567-88e6-4585-a6d6-78a4736f7230'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:43.326964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653ac482514b4668860268ef8a0a0701', 'request_id': 'req-1061c567-88e6-4585-a6d6-78a4736f7230'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:43.333097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653ac482514b4668860268ef8a0a0701', 'request_id': 'req-1061c567-88e6-4585-a6d6-78a4736f7230'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:43.346338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] The token cache is being invalidate because project 653ac482514b4668860268ef8a0a0701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:43.346620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:43.347288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1061c567-88e6-4585-a6d6-78a4736f7230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:43.348892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1098/2162] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:43 2022] DELETE /identity/v3/projects/653ac482514b4668860268ef8a0a0701 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:43.355077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.355612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.355706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.373792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:43.375023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:43.442290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1065/2163] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:43.450784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.451068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.451349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.471632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:43.472872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:43.484717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1099/2164] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:43.486362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.496791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232902fc-de7e-4ae9-8d27-28c9bdd0fe40 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.501992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.502710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.502992 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.503269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.504014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:43.504348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.506660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.521503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1066/2165] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:06:43.526667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.537299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676b522-a2cb-491e-83ec-6260925ba641 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.542526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.543351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.543697 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.544036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.544842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.546882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.579684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1100/2166] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:43 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:43.580938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.592566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193964e5-97a9-4691-b18d-8cd6578ec677 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.654261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.656117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.656117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.656117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.657318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.659331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.684609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1067/2167] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:43 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:43.687956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.700595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ac88b-2ad3-4755-a0c5-089524cccc03 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.709425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.710184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.710466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.710790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.711541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.713397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.725785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1101/2168] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:43.730421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.746080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b74818-abfc-43d8-8dce-ae54b9909489 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.755834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.757668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.758021 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.758420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.759288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.761174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.809801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.821942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1068/2169] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:43.823319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b7b7f-e264-4a70-af37-99044e5e46af admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.833404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:43.834341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.834604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.834881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] PATH_INFO: `/v3/projects/d9cf42b79fe74ab6aafd2c02734ead1d/users/fd79c24f7d4646c5ba6e66ba4e06ae05/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:43.924434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cf42b79fe74ab6aafd2c02734ead1d, user_id=fd79c24f7d4646c5ba6e66ba4e06ae05, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:43.930460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:43.932189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642957-f5e6-40d5-ae80-ff607cbb2c7a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:43.933256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:43.954651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5840833-0ee2-46f6-950c-56feab9135e8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:43.965317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1102/2170] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:43 2022] PUT /identity/v3/projects/d9cf42b79fe74ab6aafd2c02734ead1d/users/fd79c24f7d4646c5ba6e66ba4e06ae05/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:43.978354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:43.979186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:43.979715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.007468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:44.049820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] MFA Rules not processed for user `fd79c24f7d4646c5ba6e66ba4e06ae05`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:44.057822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.077824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.078793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.079201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.079540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] PATH_INFO: `/v3/users/58dd18e910a240e39c9102ff96f62c01` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.081187 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58dd18e910a240e39c9102ff96f62c01)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.083469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.185369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58dd18e910a240e39c9102ff96f62c01'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.206038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58dd18e910a240e39c9102ff96f62c01'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.223754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58dd18e910a240e39c9102ff96f62c01'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.224803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58dd18e910a240e39c9102ff96f62c01'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.226105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58dd18e910a240e39c9102ff96f62c01'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.227201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1103/2171] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:44.228488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.243344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.270964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1272cd9-5377-4ca0-a667-67d406a5adde None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.277768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.278845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.279198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.279602 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] PATH_INFO: `/v3/projects/f5a0c527e3434d9d9497960e10b2072f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.280962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a0c527e3434d9d9497960e10b2072f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.283535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.290839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1069/2172] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:43 2022] DELETE /identity/v3/users/58dd18e910a240e39c9102ff96f62c01 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:44.292769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.295465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.342458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:44.360193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a0c527e3434d9d9497960e10b2072f', 'request_id': 'req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.370646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a0c527e3434d9d9497960e10b2072f', 'request_id': 'req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.377427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b24b-664a-43df-ab57-951f2fb062b9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.384960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a0c527e3434d9d9497960e10b2072f', 'request_id': 'req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.400288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] The token cache is being invalidate because project f5a0c527e3434d9d9497960e10b2072f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:44.400500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.401251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d7c47adb-6bcf-4e54-b4cc-d1103450f14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:44.403914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1104/2173] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:44 2022] DELETE /identity/v3/projects/f5a0c527e3434d9d9497960e10b2072f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:44.405247 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac857-20b9-4ce9-9b70-28e8b473aa68 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.405732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac857-20b9-4ce9-9b70-28e8b473aa68 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.406017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac857-20b9-4ce9-9b70-28e8b473aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.425379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-038ac857-20b9-4ce9-9b70-28e8b473aa68 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:44.426668 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-038ac857-20b9-4ce9-9b70-28e8b473aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.463818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.464518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.464789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.465101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.467449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.487124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:44.524289 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1105/2174] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:44.529669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ffb7-7426-452c-bef9-690e86425a8e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.530097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ffb7-7426-452c-bef9-690e86425a8e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.530473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78ffb7-7426-452c-bef9-690e86425a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.556661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.556958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0e78ffb7-7426-452c-bef9-690e86425a8e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:44.558639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0e78ffb7-7426-452c-bef9-690e86425a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.558763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.570804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1070/2175] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:44.583224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1106/2176] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:44.586007 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.587603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.590352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.590848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.590848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.624666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:44.626184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:44.637766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1107/2177] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:44.642296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.656506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b20ace-a6ed-4166-88f2-53ef7fd5f2cd None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.663492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.664195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.664476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.664799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.665705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:44.666770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.668738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.683985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1108/2178] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:06:44.695527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.708738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404518d7-d0d8-46bf-8625-3d61cccfac22 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.709047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d066f-0037-49b3-901e-b9bb965503b6 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.716130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.716955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.717481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.717763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.719864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.720548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.720827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.721129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.722346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.726428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.741179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.742665 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.766735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1071/2179] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:44.802941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1109/2180] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:44 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:44.805065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:44.818515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6bc3f-8bb1-4fce-81b5-989bd58f737a service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:44.937115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:44.938107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:44.938455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:44.938653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:44.939352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:44.941679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:44.975401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1072/2181] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:44 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:44.977213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.001310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc072b15-431d-4920-b5c6-0888fa51c71d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.009086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.009979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.010538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.010805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.015886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.017792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.045283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1110/2182] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:45.045650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.060428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8724b2-e840-4ec4-8356-491e2815a762 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.076039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.078377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.078377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.078599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.081161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.091333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.100692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1073/2183] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:45.106142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.119967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2975f-db79-4258-8b16-9fa42cf6f340 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.134439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.135694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.136056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.136539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] PATH_INFO: `/v3/projects/9e1be884c0ff40bb9e45942dc057b54d/users/8008b430511045528f2822c77097042e/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.159593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1be884c0ff40bb9e45942dc057b54d, user_id=8008b430511045528f2822c77097042e, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.161447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.163164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec7a4-87d3-472b-8bd9-8a87fad0a147 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.163616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:45.190093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daad0ed-6f87-4761-b76d-dfa545953dc5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.191170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1111/2184] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:45 2022] PUT /identity/v3/projects/9e1be884c0ff40bb9e45942dc057b54d/users/8008b430511045528f2822c77097042e/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:45.210649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.210941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.211223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.237548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:45.237743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] MFA Rules not processed for user `8008b430511045528f2822c77097042e`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:45.238995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:45.264353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.265252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.265529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.265799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] PATH_INFO: `/v3/projects/1118eaf0b9d94db09dda81f5354c9659` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.266975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1118eaf0b9d94db09dda81f5354c9659)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.268895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.275131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:45.313923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1112/2185] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:06:45.325346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1118eaf0b9d94db09dda81f5354c9659', 'request_id': 'req-485e73c0-fb84-44b0-b9bc-a80f53f6424f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.326208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.327659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:45.334919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1118eaf0b9d94db09dda81f5354c9659', 'request_id': 'req-485e73c0-fb84-44b0-b9bc-a80f53f6424f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.342206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1118eaf0b9d94db09dda81f5354c9659', 'request_id': 'req-485e73c0-fb84-44b0-b9bc-a80f53f6424f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.360529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] The token cache is being invalidate because project 1118eaf0b9d94db09dda81f5354c9659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:45.360822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.361722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.363970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1074/2186] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:45 2022] DELETE /identity/v3/projects/1118eaf0b9d94db09dda81f5354c9659 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:45.368367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.370787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:45.401343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb55b4-38aa-4f5c-a2f0-60c610978f1d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.443980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e73c0-fb84-44b0-b9bc-a80f53f6424f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.450484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.451546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.451925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.452285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] PATH_INFO: `/v3/users/f9d00ceb38464f109591a740789cc9e5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.453775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d00ceb38464f109591a740789cc9e5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.455766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.456861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:45.457684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:45.457979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:45.458244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:45.460305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:45.512533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d00ceb38464f109591a740789cc9e5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.521446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d00ceb38464f109591a740789cc9e5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.533087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d00ceb38464f109591a740789cc9e5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.533803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d00ceb38464f109591a740789cc9e5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.534523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c7e22f83-5121-4928-88f6-a9ace1bd703d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d00ceb38464f109591a740789cc9e5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:45.540540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:45.542154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:45.543965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:45.555467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1113/2187] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:06:45.560126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1075/2188] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:45 2022] DELETE /identity/v3/users/f9d00ceb38464f109591a740789cc9e5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:45.906274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:45.930254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea182-112c-487b-992f-fccaa86d70f2 service glance] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:45.984338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:46.007246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.008254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.008611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.008959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] PATH_INFO: `/v3/projects/020d21b3593b451ba5b92e800fca021b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.010383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] RBAC: Authorizing `identity:delete_project(project_id=020d21b3593b451ba5b92e800fca021b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.013473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.075317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020d21b3593b451ba5b92e800fca021b', 'request_id': 'req-d112ce8b-2e4b-4114-a9af-25cd94c148be'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:46.084633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020d21b3593b451ba5b92e800fca021b', 'request_id': 'req-d112ce8b-2e4b-4114-a9af-25cd94c148be'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:46.101210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020d21b3593b451ba5b92e800fca021b', 'request_id': 'req-d112ce8b-2e4b-4114-a9af-25cd94c148be'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:46.121214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] The token cache is being invalidate because project 020d21b3593b451ba5b92e800fca021b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:46.121835 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:46.122242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-d112ce8b-2e4b-4114-a9af-25cd94c148be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:46.124509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1114/2189] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:45 2022] DELETE /identity/v3/projects/020d21b3593b451ba5b92e800fca021b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:46.129958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.130308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.130532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.163419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:46.164661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:46.270299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1076/2190] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:46.281198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.281678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.282015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.311148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:46.312397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:46.323426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1115/2191] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:46.327986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.339689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e1857-1de2-4769-b7b3-ea77aed13a04 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.345725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.346936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.346936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.346936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.347850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:46.348175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.349892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.375140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1077/2192] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:06:46.376401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.403759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9591c-6113-436e-9c44-8f11de326432 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.412733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.413791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.414692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.415290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.416210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.418169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.463575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1116/2193] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:46.469309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.482108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6710e0-684e-4737-b822-2b538be314f8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.548751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.549691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.549973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.550250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.550951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.553249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.578656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1078/2194] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:46.582942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.597123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef583d9-cf1b-4008-8da1-c44770c33f30 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.602894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.604011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.604011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.604263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.605388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.606997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.626978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1117/2195] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:46.628493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.640959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5411c-6e54-43c6-bb79-8608bbe39c57 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.652609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.653470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.653848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.654132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.654903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.657849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716ded2-ce3f-468e-86be-5f368097be25 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.671027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1079/2196] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:46.675125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.686994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5499a-5e6f-486b-b8e0-275fbcf1c8b1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.691715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.692627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.692930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.693241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] PATH_INFO: `/v3/projects/670a1bd6c3114c0e89da1e530ca2a36d/users/8c7286f01ae0466e90169d4af300581c/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.714063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=670a1bd6c3114c0e89da1e530ca2a36d, user_id=8c7286f01ae0466e90169d4af300581c, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.721220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:46.740803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1118/2197] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:46 2022] PUT /identity/v3/projects/670a1bd6c3114c0e89da1e530ca2a36d/users/8c7286f01ae0466e90169d4af300581c/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:46.752507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.752813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.753133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.798024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] MFA Rules not processed for user `8c7286f01ae0466e90169d4af300581c`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:46.799259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:46.849011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:46.885900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1080/2198] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:46.890399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:46.969402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27751f-e8f3-42a8-b987-9b7673ea4cdf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:46.975991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:46.976605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:46.976885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:46.977186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:46.977886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:46.979879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.002555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:47.013974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1119/2199] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:46 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:47.015145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.026316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04c2db-a92b-4719-8d19-5de76aeac222 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.083726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.084401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.084674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.084933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.085651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.087618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.112401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1081/2200] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:47 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:06:47.116501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.128319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85947403-7dec-485d-af07-85942b9c3810 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.133877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.134658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.134947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.135245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.135989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.137659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.149050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1120/2201] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:47.153439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.163924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e484a-b3f5-47c5-9f4d-cbdffd5672ff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.169362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.170269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.170543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.170807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] PATH_INFO: `/v3/projects/e52821064fdd48198044338e854b55c7/users/bf98c7601ad44772a9c9569a053a72c0/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.186216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52821064fdd48198044338e854b55c7, user_id=bf98c7601ad44772a9c9569a053a72c0, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.188933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.207870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1082/2202] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:47 2022] PUT /identity/v3/projects/e52821064fdd48198044338e854b55c7/users/bf98c7601ad44772a9c9569a053a72c0/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:47.211929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.222505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83128f75-235c-4f0b-9b44-3d1e84df9a8f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.278194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.278870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.279151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.279424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.280159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.281805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f26626-61d4-408b-b767-f5a1a7d14c71 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.291807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1121/2203] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:47.296748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.311508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51e6f-541e-4735-a2fd-9338ed93612d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.317323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.318214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.318486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.318743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] PATH_INFO: `/v3/projects/e52821064fdd48198044338e854b55c7/users/bf98c7601ad44772a9c9569a053a72c0/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.321048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52821064fdd48198044338e854b55c7, user_id=bf98c7601ad44772a9c9569a053a72c0, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.324185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.335274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1083/2204] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:47 2022] PUT /identity/v3/projects/e52821064fdd48198044338e854b55c7/users/bf98c7601ad44772a9c9569a053a72c0/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:47.346072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.346367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.347063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.377718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] MFA Rules not processed for user `bf98c7601ad44772a9c9569a053a72c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:47.379061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:47.468654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1122/2205] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:47.480610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.482364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:47.549265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510171d-d457-444a-a516-a02dfe8af4e1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.553038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.553694 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.553978 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.554254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.563589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.565063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e30f6a-1b1c-43cb-b355-ab5544c5f704 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.575843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1084/2206] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:47.652222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.731725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8a3c-fece-4b11-acf3-2fa31761ab1c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:47.736660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:47.737642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.737937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.738202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] PATH_INFO: `/v3/users/8c7286f01ae0466e90169d4af300581c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.739498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c7286f01ae0466e90169d4af300581c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:47.741542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:47.753936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:47.800693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c7286f01ae0466e90169d4af300581c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:47.809368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c7286f01ae0466e90169d4af300581c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:47.817226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c7286f01ae0466e90169d4af300581c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:47.817975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c7286f01ae0466e90169d4af300581c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:47.818817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c7286f01ae0466e90169d4af300581c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:47.842097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1123/2207] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:47 2022] DELETE /identity/v3/users/8c7286f01ae0466e90169d4af300581c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:47.846617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:47.846912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:47.847191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:47.864725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:47.866006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:47.900313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:47.936879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1085/2208] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:47.946845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:47.948459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:48.017151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac5940-65e9-405d-a547-57604ad71d83 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:48.021421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:48.022216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:48.022495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:48.022796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:48.032718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:48.034236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:48.047075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1124/2209] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:48.357545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:48.359241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:48.377708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2302210-c36e-40a9-9ad0-29d788ecaeb9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:48.390449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:48.391363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:48.391641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:48.391934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] PATH_INFO: `/v3/projects/670a1bd6c3114c0e89da1e530ca2a36d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:48.393191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=670a1bd6c3114c0e89da1e530ca2a36d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:48.395159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:48.446164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670a1bd6c3114c0e89da1e530ca2a36d', 'request_id': 'req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.452537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670a1bd6c3114c0e89da1e530ca2a36d', 'request_id': 'req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.458851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670a1bd6c3114c0e89da1e530ca2a36d', 'request_id': 'req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.472265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] The token cache is being invalidate because project 670a1bd6c3114c0e89da1e530ca2a36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:48.472556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.473240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.475051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1086/2210] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:48 2022] DELETE /identity/v3/projects/670a1bd6c3114c0e89da1e530ca2a36d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:48.483093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:48.492920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d44be-1394-4e95-b892-786f3e8536db service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:48.553062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:48.553868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:48.554145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:48.554416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] PATH_INFO: `/v3/users/bf98c7601ad44772a9c9569a053a72c0` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:48.555663 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf98c7601ad44772a9c9569a053a72c0)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:48.557637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:48.597953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:48.612542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf98c7601ad44772a9c9569a053a72c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.620579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf98c7601ad44772a9c9569a053a72c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.627443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf98c7601ad44772a9c9569a053a72c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.628096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf98c7601ad44772a9c9569a053a72c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.628710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-31a49078-25af-41b3-8c80-71d499c023b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf98c7601ad44772a9c9569a053a72c0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:48.646209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1125/2211] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:06:48 2022] DELETE /identity/v3/users/bf98c7601ad44772a9c9569a053a72c0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:48.942008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:48.943535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:48.959355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc70f6-0615-42fd-95db-4f9f82ee6c3d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.021713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.022584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.022884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.023129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] PATH_INFO: `/v3/projects/e52821064fdd48198044338e854b55c7` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.024286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52821064fdd48198044338e854b55c7)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.026371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.076448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52821064fdd48198044338e854b55c7', 'request_id': 'req-cf339ff7-7efa-468c-8bf6-5aede416ed61'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:49.082725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52821064fdd48198044338e854b55c7', 'request_id': 'req-cf339ff7-7efa-468c-8bf6-5aede416ed61'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:49.087467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:49.088891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52821064fdd48198044338e854b55c7', 'request_id': 'req-cf339ff7-7efa-468c-8bf6-5aede416ed61'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:49.104078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] The token cache is being invalidate because project e52821064fdd48198044338e854b55c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:06:49.104586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:49.105140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cf339ff7-7efa-468c-8bf6-5aede416ed61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:06:49.106988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1087/2212] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:06:48 2022] DELETE /identity/v3/projects/e52821064fdd48198044338e854b55c7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:49.111989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.112577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.112577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.131884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:49.133158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:49.198821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1126/2213] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:49.206845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.207137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.207411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.227509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:49.228895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:49.238110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1088/2214] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:49.242496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.252361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c6da3-044b-49c3-bc92-48424aa82177 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.257101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.257771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.258050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.258323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.259059 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:06:49.259394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.261128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.277548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1127/2215] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:06:49.279658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.298681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199c9ea-b650-481b-ac4e-2a11f5a6f666 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.307883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.308557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.308838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.309146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.309845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.311851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.350229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1089/2216] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:49.355744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.365822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef3740-e758-472d-b963-c0fcad327632 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.430249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.431102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.431516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.431863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.432932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.435742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.458783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1128/2217] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:49.463605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.474996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8f5a2-865e-4747-9845-c2a46a5e3835 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.480445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.481141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.481421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.481689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.482410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.484026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.495562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1090/2218] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:49.499458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.512343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d9db5-af7d-43c9-9d85-f141b5148baa admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.517642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.518335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.518614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.518882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.519598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.521240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c58171-60ce-4bc0-a0c3-271e20be9f39 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.532703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1129/2219] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:49.537047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.547418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf7d-90d2-4b45-98c9-a3b86df0f3fe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.556193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.557742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.558963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.559419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] PATH_INFO: `/v3/projects/72c1032abf9a4933b093571a3ec47626/users/dafb4efd3fe54101a1f53af8e1bdae6c/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.579976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c1032abf9a4933b093571a3ec47626, user_id=dafb4efd3fe54101a1f53af8e1bdae6c, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.582677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.597674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1091/2220] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:49 2022] PUT /identity/v3/projects/72c1032abf9a4933b093571a3ec47626/users/dafb4efd3fe54101a1f53af8e1bdae6c/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:06:49.611316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.611606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.611887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.640535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] MFA Rules not processed for user `dafb4efd3fe54101a1f53af8e1bdae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:49.641791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:49.646542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:49.713760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1130/2221] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:49.718956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.790868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f840e8-6b49-4979-b9ec-d42b5f5ad65e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.796659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.797610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.797896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.798169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.798988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.801351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.831124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1092/2222] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:49.835843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.849459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa6cc4-cdc9-4d2a-8150-7e8d9c870640 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.914839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.915587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.915913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.916193 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.917417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.919444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.943302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1131/2223] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:06:49 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:06:49.944625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.955746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b6a0d-0458-4dae-a76b-148d6221f20e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.961759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.962399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.962707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.962984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:49.963754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:49.965774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:49.976815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1093/2224] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:06:49.981225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:49.991638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a381cd-9be6-4c76-b457-c6850e40a373 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:49.997257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:49.998147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:49.998428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:49.998703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] PATH_INFO: `/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251/users/a20597e92eb342b8aff194bdf8cbc2e5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.014390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5f6442bcdc43eaa72eaed8d7d8c251, user_id=a20597e92eb342b8aff194bdf8cbc2e5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:50.017112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:50.031828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1132/2225] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:49 2022] PUT /identity/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251/users/a20597e92eb342b8aff194bdf8cbc2e5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:50.035986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:50.047242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397d49-f38f-4907-866d-ce3cfce9e1cc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:50.108775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:50.109408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:50.109691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:50.109963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.110697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:50.112316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525c036-acce-49b9-ab68-451698a4313c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:50.114107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:50.124555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1094/2226] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:06:50.128372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:50.139187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d35748-3c16-4d3f-bb4b-ca7428b3a5bc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:50.144064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:50.144955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:50.145257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:50.145535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] PATH_INFO: `/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251/users/a20597e92eb342b8aff194bdf8cbc2e5/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.147912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5f6442bcdc43eaa72eaed8d7d8c251, user_id=a20597e92eb342b8aff194bdf8cbc2e5, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:50.150635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:50.161755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1133/2227] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:06:50 2022] PUT /identity/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251/users/a20597e92eb342b8aff194bdf8cbc2e5/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:06:50.171945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:50.172238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:50.172512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.201340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] MFA Rules not processed for user `a20597e92eb342b8aff194bdf8cbc2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:06:50.202686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:50.271412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1095/2228] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:06:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:06:50.283066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:50.284935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:50.347852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026363c-cc02-43de-b6aa-b9ed527959b4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:50.351772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:50.352429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:50.352709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:50.353033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.363699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:50.365079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:50.371906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:06:50.377489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1134/2229] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:06:50.424251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:06:50.425780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:06:50.438819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c02ea-c5da-4340-a31a-b26b148b2483 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:06:50.442471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:06:50.443140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:06:50.443423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:06:50.443697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:06:50.513553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:06:50.515031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a764e21-0761-4f2b-8451-6ef244e34c10 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:06:50.527106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1096/2230] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:00.521954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=64 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:01.290966 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:01.402649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73787918-9122-4b81-ad0b-c9fa22e31361 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:01.408317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:01.409344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:01.409626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:01.409896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] PATH_INFO: `/v3/users/dafb4efd3fe54101a1f53af8e1bdae6c` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:01.411096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dafb4efd3fe54101a1f53af8e1bdae6c)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:01.413142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:01.466676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dafb4efd3fe54101a1f53af8e1bdae6c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:01.475664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dafb4efd3fe54101a1f53af8e1bdae6c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:01.482989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dafb4efd3fe54101a1f53af8e1bdae6c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:01.483654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dafb4efd3fe54101a1f53af8e1bdae6c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:01.484262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dafb4efd3fe54101a1f53af8e1bdae6c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:01.501466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1135/2231] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:01 2022] DELETE /identity/v3/users/dafb4efd3fe54101a1f53af8e1bdae6c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:01.506056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:01.506345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:01.506656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:01.558444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:01.559636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:01.629817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1097/2232] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:01.639681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:01.641341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:01.684275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:01.748128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c81bbe-e38d-41f8-8473-4de9a87bb5e1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:01.752255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:01.752939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:01.753235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:01.753511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:01.763535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:01.765328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:01.781457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1136/2233] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:02.102730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:02.104658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:02.119042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7abdea-3cbe-4209-8bd9-afcd0cbfaaad None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:02.123759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:02.124716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:02.125166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:02.125448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] PATH_INFO: `/v3/projects/72c1032abf9a4933b093571a3ec47626` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:02.127152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c1032abf9a4933b093571a3ec47626)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:02.130430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:02.189808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c1032abf9a4933b093571a3ec47626', 'request_id': 'req-2fa142b9-206a-4f3c-afbb-865254e56053'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.196857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c1032abf9a4933b093571a3ec47626', 'request_id': 'req-2fa142b9-206a-4f3c-afbb-865254e56053'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.202939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c1032abf9a4933b093571a3ec47626', 'request_id': 'req-2fa142b9-206a-4f3c-afbb-865254e56053'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.217495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] The token cache is being invalidate because project 72c1032abf9a4933b093571a3ec47626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:02.217495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.218596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.221845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1098/2234] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:02 2022] DELETE /identity/v3/projects/72c1032abf9a4933b093571a3ec47626 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:02.225154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:02.243622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeea7b2-0b3a-4d9f-9312-f2fb42a6956f service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:02.325454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:02.326393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:02.326675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:02.326979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] PATH_INFO: `/v3/users/a20597e92eb342b8aff194bdf8cbc2e5` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:02.329199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20597e92eb342b8aff194bdf8cbc2e5)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:02.331837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:02.400994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20597e92eb342b8aff194bdf8cbc2e5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.410345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20597e92eb342b8aff194bdf8cbc2e5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.417490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20597e92eb342b8aff194bdf8cbc2e5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.418100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20597e92eb342b8aff194bdf8cbc2e5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.418711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-25cf22ce-cd7d-407e-ba37-3a5380f1e07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20597e92eb342b8aff194bdf8cbc2e5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.436892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1137/2235] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:02 2022] DELETE /identity/v3/users/a20597e92eb342b8aff194bdf8cbc2e5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:02.717279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:02.718996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:02.732388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa142b9-206a-4f3c-afbb-865254e56053 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:02.786106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:02.795650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:02.796854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:02.797175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:02.797464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] PATH_INFO: `/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:02.798891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5f6442bcdc43eaa72eaed8d7d8c251)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:02.801026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:02.850180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5f6442bcdc43eaa72eaed8d7d8c251', 'request_id': 'req-9df7b715-98f1-4e83-82ef-b62494789157'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.856320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5f6442bcdc43eaa72eaed8d7d8c251', 'request_id': 'req-9df7b715-98f1-4e83-82ef-b62494789157'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.862134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5f6442bcdc43eaa72eaed8d7d8c251', 'request_id': 'req-9df7b715-98f1-4e83-82ef-b62494789157'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.875552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] The token cache is being invalidate because project fb5f6442bcdc43eaa72eaed8d7d8c251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:02.875835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.876564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9df7b715-98f1-4e83-82ef-b62494789157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:02.878161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1099/2236] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:02 2022] DELETE /identity/v3/projects/fb5f6442bcdc43eaa72eaed8d7d8c251 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:02.883120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:02.883465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:02.883691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:02.923032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:02.924263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:02.929095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:02.987940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1138/2237] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:02.995793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:02.996060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:02.996392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.015866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:03.017115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:03.025660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1100/2238] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:03.029900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.039893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d3857-418d-47e2-b1d0-295719b2739a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.044590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.045295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.045583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.045916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.046660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:03.046973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.048589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.060671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1139/2239] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:07:03.066127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.075616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5850-82ad-4ae6-bb87-4f5e0868af02 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.081719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.082352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.082661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.083083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.083776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.085799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.114918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1101/2240] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:03.118619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.129957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee7bf6-543c-448e-932d-9bf89ba247a0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.190521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.191098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.191383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.191658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.192360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.194344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.215879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1140/2241] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:03.220002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.231296 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246af319-215e-4f31-904d-073b927ae2d3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.237165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.237835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.238116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.238388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.239104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.240728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.252174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1102/2242] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:03.256377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.267093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09e70-ed26-4328-b677-c9c35c0ce6d8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.272371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.273095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.273379 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.273654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.274385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.276047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffa0a5-d313-4085-92b6-dd9c0aae9252 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.287158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1141/2243] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:03.291411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.301403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba24b-894b-441f-887c-30ba604fa35f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.305950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.306835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.307111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.307381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] PATH_INFO: `/v3/projects/37e1b6e722ee475aab36559de30ea15a/users/987a3e52a89c421bb7e23fefcac8c8b0/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.322454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e1b6e722ee475aab36559de30ea15a, user_id=987a3e52a89c421bb7e23fefcac8c8b0, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.325174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.341317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1103/2244] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:03 2022] PUT /identity/v3/projects/37e1b6e722ee475aab36559de30ea15a/users/987a3e52a89c421bb7e23fefcac8c8b0/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:03.353979 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.354492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.354614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.385890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] MFA Rules not processed for user `987a3e52a89c421bb7e23fefcac8c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:03.387493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:03.459583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1142/2245] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:03.464098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.526333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95464f3-777c-4a33-b6de-822cf70fd762 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.531569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.532234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.532518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.532795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.533516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.535447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.566303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1104/2246] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:03.570394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.577626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:03.582639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2dec-cc39-4af1-b366-1635a7a9dbc3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.644844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.645539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.645818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.646089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.646786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.648718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.669950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1143/2247] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:03.675501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.690343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d094986-7dbe-476d-bb87-7d506daa25ff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.696338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.697032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.697314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.697584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.698312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.699943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.702404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:03.712862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1105/2248] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:07:03.718189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.728649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f702d-cd95-452e-92f2-42f82d2a0ab6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.733826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.734714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.734992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.735271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] PATH_INFO: `/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4/users/2ed8f154c2de4be2a0d92ad5631149ac/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.750669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4eaa8b8a4c478ab891e72739480ea4, user_id=2ed8f154c2de4be2a0d92ad5631149ac, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.753398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.769084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1144/2249] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:03 2022] PUT /identity/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4/users/2ed8f154c2de4be2a0d92ad5631149ac/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:03.773064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.788154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b621-fc1c-423f-a902-f2ef17dabf50 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.849816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.850485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.850771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.851047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.851776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.853449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b6b5c-f2e4-4ee4-a308-710949bdf6e4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.864099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1106/2250] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:03.870369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:03.882184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7906e-6699-4ccd-a292-88a66ee41a8c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:03.888667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:03.889591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.889857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.890107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] PATH_INFO: `/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4/users/2ed8f154c2de4be2a0d92ad5631149ac/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.892727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4eaa8b8a4c478ab891e72739480ea4, user_id=2ed8f154c2de4be2a0d92ad5631149ac, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:03.895492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:03.909117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1145/2251] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:03 2022] PUT /identity/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4/users/2ed8f154c2de4be2a0d92ad5631149ac/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:03.919525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:03.919817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:03.920097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:03.958172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] MFA Rules not processed for user `2ed8f154c2de4be2a0d92ad5631149ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:03.959403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:04.041849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1107/2252] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:04.054042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:04.057335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:04.124544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1e5ca-0bee-48a8-b6e5-f9456cf95f08 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:04.128563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:04.129232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:04.129513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:04.129787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:04.138902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:04.140419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed682f8-0e9b-4ad9-b2f5-5f8cc32fc00e service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:04.150062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1146/2253] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:04.249360 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:04.259531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cde44-d3e0-46a1-9d72-97e7e410f693 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:04.263490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:04.264127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:04.264406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:04.264842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:04.323117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:04.324589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:04.333675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1108/2254] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:10.061497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9062bbf-076f-4f0e-bc06-af72e299e93f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:10.061949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9062bbf-076f-4f0e-bc06-af72e299e93f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:10.062056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9062bbf-076f-4f0e-bc06-af72e299e93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:10.089760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c9062bbf-076f-4f0e-bc06-af72e299e93f None None] MFA Rules not processed for user `2a54e9db924c440da63c04052fe058e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:10.091365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c9062bbf-076f-4f0e-bc06-af72e299e93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:10.176969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1147/2255] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:11.249500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:11.269565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:11.317578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec6ed-4f7a-4e80-b481-d262947c739a service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:11.322274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:11.323158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:11.323439 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:11.323710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] PATH_INFO: `/v3/users/987a3e52a89c421bb7e23fefcac8c8b0` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:11.324950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=987a3e52a89c421bb7e23fefcac8c8b0)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:11.326924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:11.374674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987a3e52a89c421bb7e23fefcac8c8b0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.381566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987a3e52a89c421bb7e23fefcac8c8b0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.389478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987a3e52a89c421bb7e23fefcac8c8b0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.390214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987a3e52a89c421bb7e23fefcac8c8b0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.390816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987a3e52a89c421bb7e23fefcac8c8b0'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.408927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1109/2256] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:11 2022] DELETE /identity/v3/users/987a3e52a89c421bb7e23fefcac8c8b0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:11.413884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:11.414398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:11.414509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:11.421598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:11.436590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:11.437976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:11.499910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1148/2257] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:11.510023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:11.511515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:11.571160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96444-6140-4224-bdf8-56f573656e1b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:11.575206 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:11.575852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:11.576155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:11.576426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:11.587490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:11.589695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:11.600326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1110/2258] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:11.887344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:11.888889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:11.905540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70ec3e-eb1c-4a37-a399-a9890c1bf1af None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:11.910842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:11.911888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:11.912269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:11.912641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] PATH_INFO: `/v3/projects/37e1b6e722ee475aab36559de30ea15a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:11.914034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e1b6e722ee475aab36559de30ea15a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:11.916252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:11.976165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e1b6e722ee475aab36559de30ea15a', 'request_id': 'req-928106d5-79e7-49d0-81b3-72506bb4fbc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.984763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e1b6e722ee475aab36559de30ea15a', 'request_id': 'req-928106d5-79e7-49d0-81b3-72506bb4fbc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:11.992606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e1b6e722ee475aab36559de30ea15a', 'request_id': 'req-928106d5-79e7-49d0-81b3-72506bb4fbc3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.006001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] The token cache is being invalidate because project 37e1b6e722ee475aab36559de30ea15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:12.006290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.006931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.008537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1149/2259] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:11 2022] DELETE /identity/v3/projects/37e1b6e722ee475aab36559de30ea15a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:12.012753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:12.023727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69d81-0b2f-4e12-b284-4dd5d1fd5e23 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:12.068177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:12.087671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:12.088561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.088841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.089142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] PATH_INFO: `/v3/users/2ed8f154c2de4be2a0d92ad5631149ac` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.090706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed8f154c2de4be2a0d92ad5631149ac)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:12.092631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:12.151711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed8f154c2de4be2a0d92ad5631149ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.162009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed8f154c2de4be2a0d92ad5631149ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.171460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed8f154c2de4be2a0d92ad5631149ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.172087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed8f154c2de4be2a0d92ad5631149ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.172775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ac3aa0c6-b478-4142-a468-a20879e299d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed8f154c2de4be2a0d92ad5631149ac'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.193164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1111/2260] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:12 2022] DELETE /identity/v3/users/2ed8f154c2de4be2a0d92ad5631149ac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:12.485397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:12.488110 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:12.503043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928106d5-79e7-49d0-81b3-72506bb4fbc3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:12.566297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:12.567254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.567568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.567847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] PATH_INFO: `/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.569222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4eaa8b8a4c478ab891e72739480ea4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:12.571229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:12.606863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:12.622492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4eaa8b8a4c478ab891e72739480ea4', 'request_id': 'req-4c42f959-ea11-4b1f-b912-8ea18ffe3167'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.629156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4eaa8b8a4c478ab891e72739480ea4', 'request_id': 'req-4c42f959-ea11-4b1f-b912-8ea18ffe3167'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.636525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4eaa8b8a4c478ab891e72739480ea4', 'request_id': 'req-4c42f959-ea11-4b1f-b912-8ea18ffe3167'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.653842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] The token cache is being invalidate because project aa4eaa8b8a4c478ab891e72739480ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:12.654144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.654792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-4c42f959-ea11-4b1f-b912-8ea18ffe3167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:12.656480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1150/2261] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:12 2022] DELETE /identity/v3/projects/aa4eaa8b8a4c478ab891e72739480ea4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:12.661628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.662207 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.662309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.694729 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:12.696233 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:12.764551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1112/2262] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:07:12.773517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.773809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.774098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.792838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:12.794099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:12.802332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1151/2263] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:12.806613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:12.816766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c77a9f-2e4a-4143-98a5-9327adfb153b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:12.822034 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:12.822728 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.823009 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.823279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.824057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:12.824363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:12.826029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:12.838848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1113/2264] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:07:12.843684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:12.854112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadfa0e-8f0b-47fb-a55d-9fe444cbfe90 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:12.859381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:12.860025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.860369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.860597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.861353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:12.864458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:12.897614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1152/2265] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:12 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:12.901865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:12.913449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f132b-345b-4968-9db8-ffc9b4565c62 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:12.948156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:12.974065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:12.974753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:12.975035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:12.975319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:12.976024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:12.978019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.001553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1114/2266] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:12 2022] POST /identity/v3/users => generated 305 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:13.005885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.016540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88bac1-9b08-4271-9ec7-e8bcc4c9cab7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.021328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.022021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.022299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.022569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.023303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.024969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.037225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1153/2267] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:13.041418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.053117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811b157-a0a6-48a7-ba48-1d705fb2ee9f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.058230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.058907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.059417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.059622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.060409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.062116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22be528-24c7-4664-b295-2cc77109034b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.078203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1115/2268] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:13.088314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.101748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6b46d-a23b-4a06-842f-346ef512fd3c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.106923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.107987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.108365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.108738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] PATH_INFO: `/v3/projects/5a3cca921e1a4cc180946923e0ca28ba/users/ae090ee93930419f948d6de142b1f134/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.127067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3cca921e1a4cc180946923e0ca28ba, user_id=ae090ee93930419f948d6de142b1f134, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.129920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.152150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1154/2269] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:13 2022] PUT /identity/v3/projects/5a3cca921e1a4cc180946923e0ca28ba/users/ae090ee93930419f948d6de142b1f134/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:13.158313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.158313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.158709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.207254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] MFA Rules not processed for user `ae090ee93930419f948d6de142b1f134`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:13.208469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:13.278847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1116/2270] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:13.284108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.364428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fdaaf-869b-4ec9-ab15-ddedee88e51a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.371313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.372022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.372284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.372560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.373361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.375592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.413728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1155/2271] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:13 2022] POST /identity/v3/projects => generated 407 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:13.419497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.433345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe6b44-9e6a-4d51-80d1-e83344d02d83 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.517461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.517803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.518750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.519333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.519698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.520271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:13.520753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.523298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.547640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1117/2272] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:13 2022] POST /identity/v3/users => generated 310 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:13.552411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.564816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ecf11-aa1b-4696-9862-bd6782543433 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.570767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.571447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.571727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.572000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.572732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.574376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.578894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:13.588835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1118/2273] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:07:13.589820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.597810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d984f3-ded6-4bd5-b4f1-f5603a9466fd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.601603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684aabc4-549e-401d-af7f-9b92d5e7c67e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.602525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.603313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.603692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.604063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.606704 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:13.607393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.608293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.608573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.608847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] PATH_INFO: `/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d/users/3b2d1b72a1614757803c011bbe3d092a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.629583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dd8b55c9ef4f3ca8be21fbd24c9d0d, user_id=3b2d1b72a1614757803c011bbe3d092a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.632283 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.643931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:13.649314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1119/2274] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:13 2022] PUT /identity/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d/users/3b2d1b72a1614757803c011bbe3d092a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:13.655720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.687819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.689924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.702381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1156/2275] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:13.724092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526fa43-7d32-464f-b2fe-727bf2476783 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.729127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.729822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.730145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.730418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.731160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.732834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6806c8-5a9d-45e8-9625-b6175cc5521f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.748591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1120/2276] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:13.750332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.774465 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b5bf-6bdc-4ea3-b20d-966c6c8fd84d service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:13.782649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:13.784499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.785049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.785555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] PATH_INFO: `/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d/users/3b2d1b72a1614757803c011bbe3d092a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.788763 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dd8b55c9ef4f3ca8be21fbd24c9d0d, user_id=3b2d1b72a1614757803c011bbe3d092a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:13.792169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:13.807459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1157/2277] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:13 2022] PUT /identity/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d/users/3b2d1b72a1614757803c011bbe3d092a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:13.821411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:13.821631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:13.821876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:13.856140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] MFA Rules not processed for user `3b2d1b72a1614757803c011bbe3d092a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:13.857567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:13.950279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1121/2278] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:13.961428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:13.963423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:14.051754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f77d5c-8154-459b-a61d-1e8c7e1d8b28 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:14.062928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:14.063793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:14.064100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:14.064381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:14.074963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:14.076335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:14.087232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1158/2279] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:14.289826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:14.300505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7686d5-1e6e-4857-a061-87824c0c9f98 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:14.305003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:14.305734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:14.306138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:14.306475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:14.388635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:14.390302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6b1ca-0186-4619-a19d-6630274f47b6 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:14.405287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1122/2280] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:15.055759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:15.057373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:15.175788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4891b95-5520-40c0-a95e-5c99c15a9a79 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:15.185039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:15.186361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:15.186361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:15.186817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] PATH_INFO: `/v3/users/fd79c24f7d4646c5ba6e66ba4e06ae05` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:15.189353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd79c24f7d4646c5ba6e66ba4e06ae05)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:15.192255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:15.222581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:15.273931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd79c24f7d4646c5ba6e66ba4e06ae05'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:15.287993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd79c24f7d4646c5ba6e66ba4e06ae05'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:15.303223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd79c24f7d4646c5ba6e66ba4e06ae05'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:15.304254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd79c24f7d4646c5ba6e66ba4e06ae05'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:15.305203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd79c24f7d4646c5ba6e66ba4e06ae05'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:15.337063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1159/2281] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:15 2022] DELETE /identity/v3/users/fd79c24f7d4646c5ba6e66ba4e06ae05 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:15.344078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:15.345503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:15.345990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:15.367556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:15.369188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:15.387975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:15.445447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1123/2282] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:15.455416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:15.457011 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:15.538930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ccda-08d8-4a6a-9adf-d6f942321247 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:15.550661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:15.551742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:15.552517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:15.553022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:15.567842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:15.569649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96916-8069-4d58-b322-f127c2bdde29 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:15.584512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1160/2283] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:16.105027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.107310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:16.142832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1710931-b6cb-476c-b7c5-34353d413ddd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.149293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.150453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.150796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.151130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] PATH_INFO: `/v3/projects/d9cf42b79fe74ab6aafd2c02734ead1d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.152566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9cf42b79fe74ab6aafd2c02734ead1d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.154918 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.207482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9cf42b79fe74ab6aafd2c02734ead1d', 'request_id': 'req-cced104f-7d86-4626-a1a6-6d3d0b1cac32'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:16.215823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9cf42b79fe74ab6aafd2c02734ead1d', 'request_id': 'req-cced104f-7d86-4626-a1a6-6d3d0b1cac32'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:16.221541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9cf42b79fe74ab6aafd2c02734ead1d', 'request_id': 'req-cced104f-7d86-4626-a1a6-6d3d0b1cac32'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:16.234831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] The token cache is being invalidate because project d9cf42b79fe74ab6aafd2c02734ead1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:16.235196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:16.235840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cced104f-7d86-4626-a1a6-6d3d0b1cac32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:16.237451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1124/2284] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:16 2022] DELETE /identity/v3/projects/d9cf42b79fe74ab6aafd2c02734ead1d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:16.244294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.245201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.245519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.263090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:16.264311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:16.326594 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1161/2285] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:16.335784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.336052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.336333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.356610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:16.357843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:16.368133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1125/2286] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:16.373986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.383200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0acdc-55fa-4521-9f78-851040aa9851 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.388067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.388831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.389221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.389565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.390376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:16.390755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.392821 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.406320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1162/2287] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:07:16.410115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.419454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbafe85-c237-44e8-9e3c-04d361dc5002 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.424507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.425189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.425472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.425739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.426414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.428592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.455684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1126/2288] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:16 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:16.459861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.470968 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1198b9d-7bf4-43e7-865c-39898d9aaa28 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.498306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:16.533709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.534401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.534686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.534987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.535682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.537679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.560861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1163/2289] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:16 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:16.565298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.576269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f886fd-2f78-4c1d-9f68-dd5e629a9d46 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.581267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.581940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.582222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.582496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.583223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.584851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.599165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1127/2290] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:16.600686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.612133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9beb3-fa0e-4d32-943c-679a1b763b9d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.617521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.618217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.618501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.618777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.619547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.621244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d260d-0a7a-405c-9695-ee31fd56a284 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.637232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1164/2291] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:16.638574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.649897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be0cf0-6093-4cb0-ab16-f784f454936a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.657356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.658276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.658562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.658823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] PATH_INFO: `/v3/projects/734fc82eeb014ad7b7bdb1c09236d44b/users/a9728aa79bb04cf58ea266102a3a5119/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.677388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734fc82eeb014ad7b7bdb1c09236d44b, user_id=a9728aa79bb04cf58ea266102a3a5119, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.680215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.696431 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1128/2292] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:16 2022] PUT /identity/v3/projects/734fc82eeb014ad7b7bdb1c09236d44b/users/a9728aa79bb04cf58ea266102a3a5119/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:16.701767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf54e0-6cca-43ed-821d-1b9df46cbb7f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.702060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf54e0-6cca-43ed-821d-1b9df46cbb7f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.702336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf54e0-6cca-43ed-821d-1b9df46cbb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.721766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3caf54e0-6cca-43ed-821d-1b9df46cbb7f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:16.722910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3caf54e0-6cca-43ed-821d-1b9df46cbb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:16.801191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1165/2293] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:16.811627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:16.814081 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:16.870613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:16.887857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae998a-24a4-4aa5-8cbb-336453179af8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:16.892999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:16.895773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:16.895773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:16.895773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:16.905859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:16.907380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:16.917724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1129/2294] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:07:20.768071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:20.768071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:20.768071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:20.795891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] MFA Rules not processed for user `a9728aa79bb04cf58ea266102a3a5119`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:20.797151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:20.872216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1166/2295] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:20.873125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:20.883935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958d7d-6284-4a69-97e7-834408bb5def service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:20.888935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:20.889600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:20.889882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:20.890151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:20.890844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:20.893174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:20.920749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1130/2296] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:20 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:07:20.925033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:20.936195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04472559-56e3-4d76-b2ca-3307efa476d9 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:20.973511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:20.998701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:20.999399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:20.999684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:20.999961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:21.000791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:21.002804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:21.028788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1167/2297] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:20 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:21.032578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:21.046882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50587a4d-509a-4993-9a4c-1732c7593acf admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:21.060348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:21.060348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:21.060348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:21.060348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:21.060982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:21.064105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:21.089636 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1131/2298] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:21.094630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:21.105143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8281fc1-48ef-4685-a42a-7e7de112c7b0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:21.112452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:21.113918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:21.114220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:21.114587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] PATH_INFO: `/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420/users/3101c00ce7f54f5a8e4fcc462158bc0b/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:21.131560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe1e6834c2f4581b3ce7a50dcada420, user_id=3101c00ce7f54f5a8e4fcc462158bc0b, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:21.134285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:21.149438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1168/2299] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:21 2022] PUT /identity/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420/users/3101c00ce7f54f5a8e4fcc462158bc0b/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:21.153832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:21.164734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ff09f-b9f3-4aa7-b314-0160baf4f35f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:21.227643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:21.228530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:21.228986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:21.229392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:21.230444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:21.232167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:21.244780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1132/2300] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:21.248880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:21.261477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6f5e-5027-4264-b2ad-29fd844b9a1f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:21.266743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:21.267773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:21.268117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:21.268464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] PATH_INFO: `/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420/users/3101c00ce7f54f5a8e4fcc462158bc0b/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:21.275611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe1e6834c2f4581b3ce7a50dcada420, user_id=3101c00ce7f54f5a8e4fcc462158bc0b, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:21.275611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6b77-e7b4-4559-9eb4-e5dc9ceee2b5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:21.286135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1169/2301] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:21 2022] PUT /identity/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420/users/3101c00ce7f54f5a8e4fcc462158bc0b/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:07:25.571230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:25.573810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:25.641854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97857c7-213d-4fce-a6c3-a1c95b8893ce admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:25.752449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:25.753430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:25.753663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:25.754111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] PATH_INFO: `/v3/users/ae090ee93930419f948d6de142b1f134` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:25.760826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae090ee93930419f948d6de142b1f134)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:25.762743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:25.849536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae090ee93930419f948d6de142b1f134'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:25.852682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:25.881927 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae090ee93930419f948d6de142b1f134'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:25.904454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae090ee93930419f948d6de142b1f134'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:25.905418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae090ee93930419f948d6de142b1f134'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:25.906311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae090ee93930419f948d6de142b1f134'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:25.946821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1133/2302] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:25 2022] DELETE /identity/v3/users/ae090ee93930419f948d6de142b1f134 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:25.951693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e3d63-0cdc-4a12-b896-4d3feff181d5 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:25.952269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e3d63-0cdc-4a12-b896-4d3feff181d5 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:25.952269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e3d63-0cdc-4a12-b896-4d3feff181d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:25.977307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-732e3d63-0cdc-4a12-b896-4d3feff181d5 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:25.978911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-732e3d63-0cdc-4a12-b896-4d3feff181d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:26.119888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1170/2303] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:26.157671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:26.159587 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:26.258749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3cc9-63b1-4bf6-ad43-2e15edfed1d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:26.263909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:26.264797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:26.265198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:26.265535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:26.278407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:26.279985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:26.304150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1134/2304] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:26.869251 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:26.869548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:26.870482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:26.905728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] MFA Rules not processed for user `3101c00ce7f54f5a8e4fcc462158bc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:26.907257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:26.939414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:26.941051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:26.962341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08349c71-b9a4-4c48-8252-15638170a37b service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:26.967770 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:26.968883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:26.969427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:26.969842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] PATH_INFO: `/v3/projects/5a3cca921e1a4cc180946923e0ca28ba` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:26.971305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3cca921e1a4cc180946923e0ca28ba)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:26.973504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:27.033785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3cca921e1a4cc180946923e0ca28ba', 'request_id': 'req-0fd2cafd-c654-4229-8f3d-7294b43eac99'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.040108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3cca921e1a4cc180946923e0ca28ba', 'request_id': 'req-0fd2cafd-c654-4229-8f3d-7294b43eac99'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.044417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1171/2305] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:27.050612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3cca921e1a4cc180946923e0ca28ba', 'request_id': 'req-0fd2cafd-c654-4229-8f3d-7294b43eac99'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.057387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:27.058818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.070427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] The token cache is being invalidate because project 5a3cca921e1a4cc180946923e0ca28ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:27.070769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.071400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.073228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1135/2306] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:26 2022] DELETE /identity/v3/projects/5a3cca921e1a4cc180946923e0ca28ba => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:27.077450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:27.078628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.081444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:27.145629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:27.163421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2cafd-c654-4229-8f3d-7294b43eac99 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:27.167140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59457648-c77e-446d-8e17-b9026f83fd05 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:27.169195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:27.169993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:27.172487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:27.172487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] PATH_INFO: `/v3/users/3b2d1b72a1614757803c011bbe3d092a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:27.172487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2d1b72a1614757803c011bbe3d092a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:27.174102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:27.230786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:27.231456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:27.231787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:27.231997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:27.233940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.244856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2d1b72a1614757803c011bbe3d092a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.258269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2d1b72a1614757803c011bbe3d092a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.272377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2d1b72a1614757803c011bbe3d092a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.273874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2d1b72a1614757803c011bbe3d092a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.274489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-40812cff-5c3f-4f0e-a17e-8773495804ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2d1b72a1614757803c011bbe3d092a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:27.320282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1136/2307] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:27 2022] DELETE /identity/v3/users/3b2d1b72a1614757803c011bbe3d092a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:27.326311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:27.327831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:27.340296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1172/2308] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:07:27.469615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:27.469936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:27.470215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:27.506588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:27.507854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.623908 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1137/2309] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:07:27.683466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:27.684890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.830363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29774fae-e1f3-453c-b206-031b18314c8b service glance] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:27.908463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:27.929452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:27.931290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:27.975214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:27.976433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:27.976733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:27.977027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:27.991484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:27.993539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03dba4-8a47-44fa-8228-0284aa56df83 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.005442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1173/2310] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:07:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:07:28.040644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a93bc7-dde7-469a-baf6-9d961c42ad30 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.045834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.047016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.047444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.047968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] PATH_INFO: `/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.049390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73dd8b55c9ef4f3ca8be21fbd24c9d0d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.051407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.053523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:28.138926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73dd8b55c9ef4f3ca8be21fbd24c9d0d', 'request_id': 'req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:28.152123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73dd8b55c9ef4f3ca8be21fbd24c9d0d', 'request_id': 'req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:28.161377 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73dd8b55c9ef4f3ca8be21fbd24c9d0d', 'request_id': 'req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:28.195100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] The token cache is being invalidate because project 73dd8b55c9ef4f3ca8be21fbd24c9d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:28.195903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:28.196777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8c0bdcdb-4e92-4a32-9f44-bfdfa0429ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:28.198616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1138/2311] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:27 2022] DELETE /identity/v3/projects/73dd8b55c9ef4f3ca8be21fbd24c9d0d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:07:28.204015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.204509 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.204606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.229549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:28.234844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:28.389081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1174/2312] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:28.398611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.398999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.399185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.436328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:28.437629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:28.457259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1139/2313] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:28.461281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.479442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda26f-fb15-49b6-9142-6d71bb3671cb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.491870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.492798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.493176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.493523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.494365 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:28.494742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.498581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.524534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1175/2314] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:07:28.529462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.542122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e59f56-3a59-4a21-b601-e32bf22cb0a9 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.554631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.555941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.556526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.556801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.562094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.564362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.627460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1140/2315] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:28 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:28.631637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.648018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3de7-0eb0-4683-838a-1ec3091bd30f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.736638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.737622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.738013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.738399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.741336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.741516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.776750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1176/2316] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:28 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:28.788684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.803546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08833-b7b8-4b7e-82ab-e7aa3239abae admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.820768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.821970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.822485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.823269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.824317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.827047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.841725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1141/2317] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:28.846209 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.857949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2f9-b448-49da-a3cd-266f27d6f880 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.864053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.865084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.865541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.866241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.867190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.869437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65b1af-37b9-4cff-8995-77146ed406bf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.882332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1177/2318] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:07:28.887705 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:28.903164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda869b9-af8c-4a3d-a041-30ca30cc68aa admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:28.909391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:28.911024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.911473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.911950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] PATH_INFO: `/v3/projects/83fe8c71511b466286c59dd6f5013e50/users/6260aa1241ef4041a88cab5faa107c7a/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:28.931567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fe8c71511b466286c59dd6f5013e50, user_id=6260aa1241ef4041a88cab5faa107c7a, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:28.934890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:28.956463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1142/2319] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:28 2022] PUT /identity/v3/projects/83fe8c71511b466286c59dd6f5013e50/users/6260aa1241ef4041a88cab5faa107c7a/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:28.967837 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048713a-b054-46bf-9892-017d79695208 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:28.968478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048713a-b054-46bf-9892-017d79695208 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:28.968939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048713a-b054-46bf-9892-017d79695208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:29.044179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d048713a-b054-46bf-9892-017d79695208 None None] MFA Rules not processed for user `6260aa1241ef4041a88cab5faa107c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:29.045775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d048713a-b054-46bf-9892-017d79695208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:29.093779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:29.265747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1178/2320] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:29.270358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:29.490537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374c443-63ff-4a2a-8527-20c2e82f5e65 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:29.504117 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:29.505363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:29.505649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:29.505921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:29.506628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:29.509396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:29.551737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1143/2321] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:29 2022] POST /identity/v3/projects => generated 379 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:29.556161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048713a-b054-46bf-9892-017d79695208 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:29.595398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048713a-b054-46bf-9892-017d79695208 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:29.766690 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:29.767726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:29.768225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:29.768868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:29.769813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:29.775855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:29.812307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1179/2322] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:29 2022] POST /identity/v3/users => generated 296 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:29.816146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:29.832303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cae5b-df15-4816-b5e2-2b9dbb8d129e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:29.863140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:29.863852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:29.864140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:29.864421 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:29.865200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:29.866865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:29.884201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:29.900714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1144/2323] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:07:29.901200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:29.916291 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067f243-a498-4cd7-8283-cc69e412f77a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:29.935570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:29.936488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:29.936773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:29.937098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] PATH_INFO: `/v3/projects/7c9aae48ff844110bc92e88a1694a8cc/users/ad3c255e21ba4d98a84cdf225eacdf3a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:29.970544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9aae48ff844110bc92e88a1694a8cc, user_id=ad3c255e21ba4d98a84cdf225eacdf3a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:29.973863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:29.997920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1180/2324] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:29 2022] PUT /identity/v3/projects/7c9aae48ff844110bc92e88a1694a8cc/users/ad3c255e21ba4d98a84cdf225eacdf3a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:29.999145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:30.016366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacb769-e133-43a9-8fcf-451bc4964d60 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:30.106004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:30.106634 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:30.106916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:30.107191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:30.107933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:30.109629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eead910-b647-40ad-b9f3-0aa361c93f24 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:30.120685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1145/2325] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:30.124926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:30.139386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2115ea0-c442-4077-aa47-43a7aaa9a68f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:30.144799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:30.145748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:30.146031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:30.156521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] PATH_INFO: `/v3/projects/7c9aae48ff844110bc92e88a1694a8cc/users/ad3c255e21ba4d98a84cdf225eacdf3a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:30.158108 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9aae48ff844110bc92e88a1694a8cc, user_id=ad3c255e21ba4d98a84cdf225eacdf3a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:30.160883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:30.174252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1181/2326] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:30 2022] PUT /identity/v3/projects/7c9aae48ff844110bc92e88a1694a8cc/users/ad3c255e21ba4d98a84cdf225eacdf3a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:07:30.186800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:30.187087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:30.187366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:30.220001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] MFA Rules not processed for user `ad3c255e21ba4d98a84cdf225eacdf3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:30.221290 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:30.312026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1146/2327] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:30.324242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:30.326510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:30.413309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:30.418694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517c094-ed81-4f4f-bec9-0562b336b557 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:30.442448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:30.443151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:30.443432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:30.443702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:30.474353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:30.475920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a8ba5-1394-408d-8c13-a43ac996ec20 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:30.489858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1182/2328] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:30.936580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:31.067111 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d8f66-183f-4543-9c18-92de6a092f01 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:31.074704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:31.075808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:31.076142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:31.076417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] PATH_INFO: `/v3/users/6260aa1241ef4041a88cab5faa107c7a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:31.078888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6260aa1241ef4041a88cab5faa107c7a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:31.085984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:31.137968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:31.200653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6260aa1241ef4041a88cab5faa107c7a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:31.209688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6260aa1241ef4041a88cab5faa107c7a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:31.223762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6260aa1241ef4041a88cab5faa107c7a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:31.224870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6260aa1241ef4041a88cab5faa107c7a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:31.226087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6260aa1241ef4041a88cab5faa107c7a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:31.250550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1147/2329] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:30 2022] DELETE /identity/v3/users/6260aa1241ef4041a88cab5faa107c7a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:31.261775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:31.262068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:31.262362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:31.304706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:31.307455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:31.414464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1183/2330] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:31.425580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:31.427121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:31.494810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620957c7-202c-4f41-97dc-4a4c01384eb2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:31.498771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:31.499511 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:31.499884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:31.500222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:31.511140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:31.512975 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:31.525189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1148/2331] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:31.965896 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:31.967332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:31.989900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ae2f8-d9a0-4058-b83e-3b65bbf09d20 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:32.001731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:32.004364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:32.005357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:32.005794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] PATH_INFO: `/v3/projects/83fe8c71511b466286c59dd6f5013e50` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:32.007471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fe8c71511b466286c59dd6f5013e50)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:32.009971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:32.083256 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fe8c71511b466286c59dd6f5013e50', 'request_id': 'req-c024a7ee-fb14-4c05-8450-2d10077ffa9f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.092618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fe8c71511b466286c59dd6f5013e50', 'request_id': 'req-c024a7ee-fb14-4c05-8450-2d10077ffa9f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.099248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fe8c71511b466286c59dd6f5013e50', 'request_id': 'req-c024a7ee-fb14-4c05-8450-2d10077ffa9f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.118178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] The token cache is being invalidate because project 83fe8c71511b466286c59dd6f5013e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:32.120344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.121417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.125008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1184/2332] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:31 2022] DELETE /identity/v3/projects/83fe8c71511b466286c59dd6f5013e50 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:32.128071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:32.139597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7ef07-6718-4871-8ed4-e5d68c1b2255 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:32.224047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:32.224982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:32.225427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:32.225701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] PATH_INFO: `/v3/users/ad3c255e21ba4d98a84cdf225eacdf3a` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:32.226928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3c255e21ba4d98a84cdf225eacdf3a)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:32.228868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:32.289437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:32.291099 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3c255e21ba4d98a84cdf225eacdf3a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.305035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3c255e21ba4d98a84cdf225eacdf3a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.324222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3c255e21ba4d98a84cdf225eacdf3a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.325130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3c255e21ba4d98a84cdf225eacdf3a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.326696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-05f6a7cd-1ef7-4d8a-9da8-b8d5e2ad41f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3c255e21ba4d98a84cdf225eacdf3a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:32.374762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1149/2333] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:32 2022] DELETE /identity/v3/users/ad3c255e21ba4d98a84cdf225eacdf3a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:07:32.993217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:32.995675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:33.020794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024a7ee-fb14-4c05-8450-2d10077ffa9f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.114597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.115820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.116248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.116678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] PATH_INFO: `/v3/projects/7c9aae48ff844110bc92e88a1694a8cc` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.121131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9aae48ff844110bc92e88a1694a8cc)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.123074 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:33.155609 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:33.265506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9aae48ff844110bc92e88a1694a8cc', 'request_id': 'req-026c3625-1a94-48a5-9bde-f3bb305653d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:33.273263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9aae48ff844110bc92e88a1694a8cc', 'request_id': 'req-026c3625-1a94-48a5-9bde-f3bb305653d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:33.289362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9aae48ff844110bc92e88a1694a8cc', 'request_id': 'req-026c3625-1a94-48a5-9bde-f3bb305653d3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:33.313211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] The token cache is being invalidate because project 7c9aae48ff844110bc92e88a1694a8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:33.313999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:33.315512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-026c3625-1a94-48a5-9bde-f3bb305653d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:33.317743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1185/2334] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:32 2022] DELETE /identity/v3/projects/7c9aae48ff844110bc92e88a1694a8cc => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:33.322678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.324776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.325364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.351058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:33.352355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:33.453738 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1150/2335] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:33.462188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.462926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.463268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.503333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:33.504810 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:33.522755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1186/2336] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:33.526395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:33.539928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f245e1f-d55b-47e6-8bd8-0999ae50ef21 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.555040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.556001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.556464 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.556890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.557815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:33.558272 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.560101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:33.593050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1151/2337] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:07:33.593880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:33.606939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2b7db-dc04-49d4-9fae-8c0464fada0a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.622458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.623604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.624044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.624479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.626020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.630617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:33.687940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1187/2338] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:33 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:07:33.692530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:33.704855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edef55d-2c9e-409b-bf6a-2efd22f0a2de admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.850806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.852019 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.852463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.853581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.857265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.860474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:33.906716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1152/2339] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:33 2022] POST /identity/v3/users => generated 282 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:07:33.907633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:33.925796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c237151-f71d-49ae-8174-2e67f2c74582 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.939401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.940358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.940878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.941400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.942367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.944257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:33.958453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1188/2340] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:33.963460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:33.981451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332dee-49b2-4a63-8afa-5b419bdcb6cc admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:33.991427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:33.992985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:33.993488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:33.993921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:33.994689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:33.996615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca99c58-0247-4f0b-9f0f-5d47cc619e9b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:34.048210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:34.061248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1153/2341] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:34.070486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5a8b-d3a0-4138-a00a-befca78d08a6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:34.081737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:34.083778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:34.085696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:34.088211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] PATH_INFO: `/v3/projects/05c0e306546442ee8675ca5ceb5acd61/users/1e096496adda485cabdf9c5815210dca/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:34.125202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c0e306546442ee8675ca5ceb5acd61, user_id=1e096496adda485cabdf9c5815210dca, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:34.128405 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:34.158630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1189/2342] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:34 2022] PUT /identity/v3/projects/05c0e306546442ee8675ca5ceb5acd61/users/1e096496adda485cabdf9c5815210dca/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:07:34.172987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61feef6-bf82-4552-ba04-6964d0096384 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:34.172987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61feef6-bf82-4552-ba04-6964d0096384 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:34.172987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61feef6-bf82-4552-ba04-6964d0096384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:34.220181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f61feef6-bf82-4552-ba04-6964d0096384 None None] MFA Rules not processed for user `1e096496adda485cabdf9c5815210dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:34.221777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f61feef6-bf82-4552-ba04-6964d0096384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:34.264463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:34.385619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1154/2343] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:34.419158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:34.421267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:34.617177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f1928-2289-42cb-8eaf-a7aef00f669b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:34.623034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:34.624411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:34.624709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:34.625569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:34.644040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:34.646282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:34.674519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1190/2344] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:40.729656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea5479-8ce1-444a-9e53-8218a4b17a96 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:40.730254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea5479-8ce1-444a-9e53-8218a4b17a96 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:40.730366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea5479-8ce1-444a-9e53-8218a4b17a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:40.749476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-54ea5479-8ce1-444a-9e53-8218a4b17a96 None None] MFA Rules not processed for user `a9728aa79bb04cf58ea266102a3a5119`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:40.750763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-54ea5479-8ce1-444a-9e53-8218a4b17a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:40.818098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1155/2345] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:41.385827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:41.387520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:41.441983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:41.466813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d0ae9-292c-496e-a9bf-ed36d234d408 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:41.472002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:41.473100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:41.473442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:41.473719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] PATH_INFO: `/v3/users/8008b430511045528f2822c77097042e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:41.474927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8008b430511045528f2822c77097042e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:41.476819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:41.527646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8008b430511045528f2822c77097042e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:41.539502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8008b430511045528f2822c77097042e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:41.548393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8008b430511045528f2822c77097042e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:41.549285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8008b430511045528f2822c77097042e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:41.550176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8008b430511045528f2822c77097042e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:41.569943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1191/2346] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:07:41 2022] DELETE /identity/v3/users/8008b430511045528f2822c77097042e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:41.574816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:41.575092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:41.575367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:41.597588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:41.598997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:41.668304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1156/2347] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:41.676428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:41.677895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:41.742281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f321e-24cc-4a55-b230-cc9e02a6d1be admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:41.746767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:41.747433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:41.747711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:41.747983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:41.758411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:41.759847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:41.775068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1192/2348] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:41.914775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:41.917177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.007504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:42.035654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01424b-946a-4d78-8d05-c36773ba3ef7 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.041765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.042428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.042711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.042972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.045170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.057659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.059273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ad8e2-a78b-41f4-aaaf-dc6ab7f0c439 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.072506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.073996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.080983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1157/2349] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:42.088150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ace720-bd90-459a-9e58-802a8d2dba79 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.094869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.096146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.096463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.097134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] PATH_INFO: `/v3/projects/9e1be884c0ff40bb9e45942dc057b54d` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.098447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1be884c0ff40bb9e45942dc057b54d)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.101855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.154809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1be884c0ff40bb9e45942dc057b54d', 'request_id': 'req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:42.161804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1be884c0ff40bb9e45942dc057b54d', 'request_id': 'req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:42.166412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:42.169603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1be884c0ff40bb9e45942dc057b54d', 'request_id': 'req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:42.185724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] The token cache is being invalidate because project 9e1be884c0ff40bb9e45942dc057b54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:07:42.185924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:42.186543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-fc46eb6b-0ae2-4cd5-9892-02220307d5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:07:42.188107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1193/2350] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:07:42 2022] DELETE /identity/v3/projects/9e1be884c0ff40bb9e45942dc057b54d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:42.193513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.193955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.194390 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.218522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:42.219737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.282062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1158/2351] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:42.290773 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.291231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.291563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.312287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:42.313553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.321886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1194/2352] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:42.326181 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.336691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58b0f-e38d-4c47-ad89-b7f38c38b6dd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.341773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.342445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.342722 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.342990 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.344165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:07:42.344165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.345638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.357785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1159/2353] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:07:42.369556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.379937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7b785-f201-4287-9d02-6846d4e54947 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.388774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.388774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.388774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.388774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.389611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.390014 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.420936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1195/2354] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:07:42 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:42.424795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.435830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa4410-6983-49c9-994a-0abf2184aa5e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.498219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.498901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.499184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.499452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.500162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.502154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.533500 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1160/2355] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:07:42 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:07:42.538133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.554640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571b463-3758-4588-b14d-9a72c22f0b5d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.560587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.561799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.561799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.561996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.562669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.566660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.581983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1196/2356] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:42.586797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.598328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eaead-a8dd-4ffc-99de-c764d2d0e1fa admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.603812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.604481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.604764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.605074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.605797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.607441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d17b84-9866-4690-9fdd-d7027205612a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.622053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1161/2357] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:07:42.623254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.634607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ebdd-7ffd-458a-a64f-0830c9d22b39 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.641034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.641034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.641034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.641034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] PATH_INFO: `/v3/projects/171326d56d234a3d90bc61643b76f113/users/b8358ccdda51447d8372602b27f43095/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.658437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=171326d56d234a3d90bc61643b76f113, user_id=b8358ccdda51447d8372602b27f43095, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.661748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.682015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1197/2358] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:07:42 2022] PUT /identity/v3/projects/171326d56d234a3d90bc61643b76f113/users/b8358ccdda51447d8372602b27f43095/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:07:42.692861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d8383-b210-49e6-9788-10ffdec6aca4 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.693183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d8383-b210-49e6-9788-10ffdec6aca4 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.693457 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d8383-b210-49e6-9788-10ffdec6aca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.722241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1f9d8383-b210-49e6-9788-10ffdec6aca4 None None] MFA Rules not processed for user `b8358ccdda51447d8372602b27f43095`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:42.723483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1f9d8383-b210-49e6-9788-10ffdec6aca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.752983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:42.796357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1162/2359] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:42.807866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:42.809673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:42.876767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f31dc-42a9-4170-9d9f-36e96830c79b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:42.880964 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:42.881653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:42.881939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:42.882214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:42.892401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:42.894198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:42.904961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1198/2360] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:07:43.194124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651deeef-d675-4e17-ac78-dbd0bae403f8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:43.195420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651deeef-d675-4e17-ac78-dbd0bae403f8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:43.195848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651deeef-d675-4e17-ac78-dbd0bae403f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:43.260028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-651deeef-d675-4e17-ac78-dbd0bae403f8 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:43.261378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-651deeef-d675-4e17-ac78-dbd0bae403f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:43.340403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1163/2361] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:43.352748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:07:43.354314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:43.449222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a2e12-6de0-4e1c-93de-7ce184981de3 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:07:43.454467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:07:43.455121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:43.455402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:43.455674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:43.462157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:43.465176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:07:43.467066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf092d2-af99-4705-b1d8-e2de681bf74d service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:07:43.476547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1199/2362] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:07:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:07:45.351578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:07:45.352150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:07:45.352150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:07:45.371484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] MFA Rules not processed for user `b8358ccdda51447d8372602b27f43095`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:07:45.372717 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:07:45.380712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1164/2363] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:07:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:07:53.481032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:07:55.365305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=57 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:22.352873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5e5d-5b9e-4044-b519-40edccc386d7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:22.353447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5e5d-5b9e-4044-b519-40edccc386d7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:22.353447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5e5d-5b9e-4044-b519-40edccc386d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:22.372144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ed6f5e5d-5b9e-4044-b519-40edccc386d7 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:22.373471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ed6f5e5d-5b9e-4044-b519-40edccc386d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:22.381437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1200/2364] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:22.393592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:22.405542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe9135-e788-4f92-a071-4dbce67ee03c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:22.410782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:22.411429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:22.411716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:22.411984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:22.420490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:22.422139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:22.430970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1165/2365] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:08:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:08:22.995675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56b6e8-1197-488d-ae4c-6b16591f2823 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:22.995959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56b6e8-1197-488d-ae4c-6b16591f2823 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:22.996231 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56b6e8-1197-488d-ae4c-6b16591f2823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.014148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4e56b6e8-1197-488d-ae4c-6b16591f2823 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:23.015338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4e56b6e8-1197-488d-ae4c-6b16591f2823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:23.023117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1201/2366] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:23.036285 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:23.046713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508d3cd-42f5-4e57-9543-d756090122d6 service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:23.052134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:23.052773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:23.053225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:23.053504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.061693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:23.063307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:23.071850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1166/2367] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:08:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:08:23.288941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef240c-2aed-457f-ad47-7aafc71dc4d4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:23.289321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef240c-2aed-457f-ad47-7aafc71dc4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:23.289698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef240c-2aed-457f-ad47-7aafc71dc4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.321334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dcef240c-2aed-457f-ad47-7aafc71dc4d4 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:23.323259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dcef240c-2aed-457f-ad47-7aafc71dc4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:23.336201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1202/2368] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:23.351776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:23.363533 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a33e-6f6b-4a32-b0d4-90bd5947184f service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:23.369803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:23.370813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:23.371175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:23.371482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.382413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:23.384037 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:23.392503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1167/2369] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:08:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:08:23.750065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:23.750570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:23.750570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.771319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:23.772535 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:23.780874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1203/2370] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:23.793219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:23.803979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edab089-1cb9-41c0-8a67-9297e9cdbd6e service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:23.810352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:23.811218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:23.811522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:23.811802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:23.821104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:23.822715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:23.831096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1168/2371] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:08:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:08:25.498124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:25.499636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:25.578666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4df7f-3444-4858-8cbf-6f69ae7e6fb4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:25.583413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:25.584280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:25.584585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:25.584861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:25.587214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:25.653138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:25.654626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db718cc3-6ec1-4936-8db2-d260135354cf service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:25.664492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1204/2372] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:25.948271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.012943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08440-eb97-439d-a8f7-56de7e0ac006 service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.018151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.019042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.019324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.019601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] PATH_INFO: `/v3/users/b8358ccdda51447d8372602b27f43095` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.021743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8358ccdda51447d8372602b27f43095)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.023706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.077785 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8358ccdda51447d8372602b27f43095'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.086071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8358ccdda51447d8372602b27f43095'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.094733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8358ccdda51447d8372602b27f43095'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.095504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8358ccdda51447d8372602b27f43095'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.096101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8358ccdda51447d8372602b27f43095'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.115413 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1169/2373] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:08:25 2022] DELETE /identity/v3/users/b8358ccdda51447d8372602b27f43095 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:26.120301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556bdce-0863-474b-aed1-3e209930408a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.120581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556bdce-0863-474b-aed1-3e209930408a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.120856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556bdce-0863-474b-aed1-3e209930408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.143918 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4556bdce-0863-474b-aed1-3e209930408a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:26.146133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4556bdce-0863-474b-aed1-3e209930408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:26.212109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1205/2374] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:08:26.222496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.224255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:26.258857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:26.290195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783281-9f89-4b54-b85d-5508551295aa admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.294701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.295394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.295666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.295930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.322428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.323907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eebdb-a37b-4a4d-b3ab-2c2ceefde0ca service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.339477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1170/2375] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:26.519159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556bdce-0863-474b-aed1-3e209930408a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.521329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4556bdce-0863-474b-aed1-3e209930408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:26.535861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556bdce-0863-474b-aed1-3e209930408a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.540860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.541977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.542435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.542696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] PATH_INFO: `/v3/projects/171326d56d234a3d90bc61643b76f113` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.544138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171326d56d234a3d90bc61643b76f113)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.546115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.570103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:26.602844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171326d56d234a3d90bc61643b76f113', 'request_id': 'req-aba5100f-bf94-46ef-a4af-86a013907004'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.610198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171326d56d234a3d90bc61643b76f113', 'request_id': 'req-aba5100f-bf94-46ef-a4af-86a013907004'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.618892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171326d56d234a3d90bc61643b76f113', 'request_id': 'req-aba5100f-bf94-46ef-a4af-86a013907004'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.638797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] The token cache is being invalidate because project 171326d56d234a3d90bc61643b76f113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:08:26.639283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.640148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aba5100f-bf94-46ef-a4af-86a013907004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:26.642135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1206/2376] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:08:26 2022] DELETE /identity/v3/projects/171326d56d234a3d90bc61643b76f113 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:26.647115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.647405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.647682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.666626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:26.667506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:26.734596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1171/2377] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:26.743217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.743507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.743851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.765593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:26.766749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:26.778093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1207/2378] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:26.782649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.794467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb735-afc8-4437-9b49-31de493e29af None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.800302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.801177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.801719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.801873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.802631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:08:26.802963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.804984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.820939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1172/2379] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:08:26.823392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.833399 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6003e1-90a6-473f-90a0-bfe285c25b98 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.839097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.839953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.840239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.840568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.841794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.843917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.871939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1208/2380] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:08:26 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:08:26.876161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.886492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ed02c-cb7f-4dcc-81d0-28ccd9afdd2a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.945598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.946550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.946953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.947235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.948029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.950313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:26.975286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1173/2381] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:08:26 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:08:26.976589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:26.987354 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3b97f-bba8-4f7f-a9cb-88b9e7dda969 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:26.994472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:26.994472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:26.994472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:26.994787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:26.995318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:26.996969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:27.009694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1209/2382] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:08:27.013913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:27.024821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a15d0a-e7d2-4df8-b469-7d23c1efc47d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:27.029954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:27.030632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.030913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.031188 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.031913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:27.033612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d05be-514f-4a2a-bb66-92301227bcf4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:27.050343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1174/2383] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:08:27.051477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:27.062522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390769-ef80-4ce3-b4c9-1f78aaa4e005 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:27.067991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:27.068951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.069232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.069503 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] PATH_INFO: `/v3/projects/bc7de1467b8b4600ae850384aa65cbe1/users/51918337524d45feb60d9706555f76c4/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.084530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7de1467b8b4600ae850384aa65cbe1, user_id=51918337524d45feb60d9706555f76c4, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:27.087255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:27.103814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1210/2384] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:08:27 2022] PUT /identity/v3/projects/bc7de1467b8b4600ae850384aa65cbe1/users/51918337524d45feb60d9706555f76c4/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:08:27.114913 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.115193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.115472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.142797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] MFA Rules not processed for user `51918337524d45feb60d9706555f76c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:27.144027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:27.169352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:27.211399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1175/2385] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:27.223086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:27.225227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:27.284076 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98ad13-dbdd-4041-807d-7425e0d6c2e3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:27.288346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:27.289009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.289287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.289558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.298339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:27.299781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453bc7-9151-4629-a221-7df3cf8d3cff service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:27.309197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1211/2386] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:27.427442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:27.430288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:27.520337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c41ea-39d6-4712-ae52-19bd9a3d2321 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:27.530182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:27.531108 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.531468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.531674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] PATH_INFO: `/v3/users/1e096496adda485cabdf9c5815210dca` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.533303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e096496adda485cabdf9c5815210dca)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:27.535288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:27.642046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e096496adda485cabdf9c5815210dca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:27.659078 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e096496adda485cabdf9c5815210dca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:27.681510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e096496adda485cabdf9c5815210dca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:27.682229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e096496adda485cabdf9c5815210dca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:27.683071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e096496adda485cabdf9c5815210dca'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:27.718852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1176/2387] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:08:27 2022] DELETE /identity/v3/users/1e096496adda485cabdf9c5815210dca => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:27.724852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:27.725873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:27.726301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:27.770575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:27.775091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:27.842989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:27.879391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1212/2388] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:27.890330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:27.892231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:27.990708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:28.017371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1e8c1-d264-4b6e-8b4e-3d3d7bba0c48 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:28.041504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:28.042193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:28.042471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:28.042741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:28.067484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:28.071242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:28.100415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1177/2389] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:28.781167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:28.789274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:28.801395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bbd35-0926-408b-9cf3-84745b816339 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:28.806515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:28.807458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:28.807804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:28.808165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] PATH_INFO: `/v3/projects/05c0e306546442ee8675ca5ceb5acd61` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:28.809458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c0e306546442ee8675ca5ceb5acd61)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:28.811464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:28.874758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c0e306546442ee8675ca5ceb5acd61', 'request_id': 'req-15090b2f-5e9d-445a-8827-d60564d6a08c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:28.886825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c0e306546442ee8675ca5ceb5acd61', 'request_id': 'req-15090b2f-5e9d-445a-8827-d60564d6a08c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:28.894943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c0e306546442ee8675ca5ceb5acd61', 'request_id': 'req-15090b2f-5e9d-445a-8827-d60564d6a08c'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:28.912115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] The token cache is being invalidate because project 05c0e306546442ee8675ca5ceb5acd61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:08:28.912375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:28.913489 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-15090b2f-5e9d-445a-8827-d60564d6a08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:28.915486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1213/2390] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:08:28 2022] DELETE /identity/v3/projects/05c0e306546442ee8675ca5ceb5acd61 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:36.742793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:36.744538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:36.758446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78fd4-9011-46a0-8b5d-83db70be14e1 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:36.820628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:36.821654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:36.821935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:36.822205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] PATH_INFO: `/v3/users/51918337524d45feb60d9706555f76c4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:36.823544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51918337524d45feb60d9706555f76c4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:36.825535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:36.874174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51918337524d45feb60d9706555f76c4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:36.881784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51918337524d45feb60d9706555f76c4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:36.890302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51918337524d45feb60d9706555f76c4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:36.891465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51918337524d45feb60d9706555f76c4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:36.892125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51918337524d45feb60d9706555f76c4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:36.910042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1178/2391] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:08:36 2022] DELETE /identity/v3/users/51918337524d45feb60d9706555f76c4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:36.914936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:36.915485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:36.915709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:36.934638 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:36.935862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:36.999782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1214/2392] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:08:37.009498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:37.010932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:37.070025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e230d0d-7d0e-4db1-bb3b-9b3f5e3f1769 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:37.074610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:37.075262 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:37.075544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:37.075819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:37.085480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:37.086928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3393f1-caee-4ec9-a693-8e4b70424b3d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:37.097529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1179/2393] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:37.405957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:37.407469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:37.419768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae15fb-9b53-40d0-babc-c5fe8a825e86 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:37.425364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:37.426266 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:37.426545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:37.426813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] PATH_INFO: `/v3/projects/bc7de1467b8b4600ae850384aa65cbe1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:37.428042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc7de1467b8b4600ae850384aa65cbe1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:37.430017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:37.481389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc7de1467b8b4600ae850384aa65cbe1', 'request_id': 'req-921ebdeb-08e6-487e-b2da-06fe9fc539e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:37.487675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc7de1467b8b4600ae850384aa65cbe1', 'request_id': 'req-921ebdeb-08e6-487e-b2da-06fe9fc539e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:37.493337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc7de1467b8b4600ae850384aa65cbe1', 'request_id': 'req-921ebdeb-08e6-487e-b2da-06fe9fc539e4'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:37.496576 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:37.511433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] The token cache is being invalidate because project bc7de1467b8b4600ae850384aa65cbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:08:37.511723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:37.512454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-921ebdeb-08e6-487e-b2da-06fe9fc539e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:08:37.514140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1215/2394] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:08:37 2022] DELETE /identity/v3/projects/bc7de1467b8b4600ae850384aa65cbe1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:08:47.092861 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:47.509197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:08:49.019086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:49.019525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:49.019619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:49.021209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1180/2395] 10.222.0.55 () {54 vars in 912 bytes} [Tue Jun 28 12:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:08:49.026492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dcf8-805a-4a24-8c3c-ccd6e3c29296 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:49.026935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dcf8-805a-4a24-8c3c-ccd6e3c29296 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:49.027094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dcf8-805a-4a24-8c3c-ccd6e3c29296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:49.048450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6735dcf8-805a-4a24-8c3c-ccd6e3c29296 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:49.049729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6735dcf8-805a-4a24-8c3c-ccd6e3c29296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:49.111698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1216/2396] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:49.124559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:49.126630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:49.144414 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f9c58-16d9-4ab6-a17a-68d1b01e8d92 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:49.149312 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:49.149956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:49.150301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:49.150579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:49.158661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:49.160114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:49.173664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1181/2397] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:08:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:08:52.218845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f24-aa0b-4d19-a71c-10d5d7310746 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:52.219322 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f24-aa0b-4d19-a71c-10d5d7310746 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:52.219440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f24-aa0b-4d19-a71c-10d5d7310746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:52.238344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-51024f24-aa0b-4d19-a71c-10d5d7310746 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:08:52.240471 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-51024f24-aa0b-4d19-a71c-10d5d7310746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:52.307358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1217/2398] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:08:52.338777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:08:52.340237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:08:52.414488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4d5c0-c201-4fec-8c37-bb6694791d46 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:08:52.419846 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:08:52.420510 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:08:52.420794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:08:52.421100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:08:52.429730 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:08:52.431343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc946ad-15fd-4fcd-b4db-ab1068bffd6c service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:08:52.439816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1182/2399] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:08:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:09:02.299650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=48 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:02.436961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=38 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:08.585497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:08.586148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:08.586148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:08.587541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1218/2400] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:09:08.593760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526aa9e-d1e6-490d-8ade-1be570383233 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:08.594221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526aa9e-d1e6-490d-8ade-1be570383233 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:08.594363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526aa9e-d1e6-490d-8ade-1be570383233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:08.646336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a526aa9e-d1e6-490d-8ade-1be570383233 None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:09:08.647592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a526aa9e-d1e6-490d-8ade-1be570383233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:08.727166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1183/2401] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:09:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:09:08.739130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:08.740961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:08.830624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ed3d1-42e0-43b5-ab55-38b889e3383c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:08.835018 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:08.835671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:08.835951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:08.836226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:08.848630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:08.850951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaadfb-26ab-4dc5-b3d2-13b80bd9bd9b service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:08.867274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1219/2402] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:08.939948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:08.940238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:08.940515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:08.941941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1184/2403] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:09:08.947180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed5e1-db37-47b7-900d-9674637be7a8 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:08.947467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed5e1-db37-47b7-900d-9674637be7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:08.947740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ed5e1-db37-47b7-900d-9674637be7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:08.967942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-662ed5e1-db37-47b7-900d-9674637be7a8 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:09:08.969180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-662ed5e1-db37-47b7-900d-9674637be7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:08.977656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1220/2404] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:09:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:09:08.989660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:08.991563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:09.006967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39898-5c12-4f5d-888a-892f4b2dcd80 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:09.011048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:09.011691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:09.011974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:09.012242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:09.019638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:09.021130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f6209-e463-4f4e-ab25-41565355e859 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:09.029182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1185/2405] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:09.113765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dbb9-ee3f-43c7-8a8e-1763a2d87875 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:09.114073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dbb9-ee3f-43c7-8a8e-1763a2d87875 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:09.114457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9dbb9-ee3f-43c7-8a8e-1763a2d87875 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:09.116423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1221/2406] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:09:09.121973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d173845-5296-4732-bdaa-770f9b3a35de None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:09.122259 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d173845-5296-4732-bdaa-770f9b3a35de None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:09.122562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d173845-5296-4732-bdaa-770f9b3a35de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:09.141214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2d173845-5296-4732-bdaa-770f9b3a35de None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:09:09.142622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2d173845-5296-4732-bdaa-770f9b3a35de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:09.152693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1186/2407] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:09:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:09:09.164141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9dbb9-ee3f-43c7-8a8e-1763a2d87875 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:09.178161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9dbb9-ee3f-43c7-8a8e-1763a2d87875 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:09.182010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:09.182657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:09.182939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:09.183212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:09.193978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:09.195421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:09.206703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1222/2408] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:11.993931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:11.994448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:11.994448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:12.015332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:09:12.017032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:12.024387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1187/2409] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:09:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:09:12.037128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:12.054944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438cf5-fa9d-45ad-a584-a8c877c7f79d service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:12.063817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:12.064276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:12.064560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:12.064834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:12.077874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:12.079926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:12.089378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1223/2410] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:09:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:09:22.009910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=58 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:22.085002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=50 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:25.197301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:25.208626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34195b28-8b8d-42a3-b559-a86409f903de None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:25.214168 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:25.214826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:25.215110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:25.215382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:25.217361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:25.231605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:25.233101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:25.245388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1188/2411] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:27.700164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:27.701838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:27.760129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ffd1a-776a-4169-9889-8dd3bb806b7e service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:27.764156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:27.764807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:27.765207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:27.765484 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:27.767475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:27.781727 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:27.783182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:27.794556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1224/2412] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:35.239584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:37.789540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=26 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:38.297424 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:38.299112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:38.320310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1a222-05bd-4a85-bf50-8284e16aeb88 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:38.332886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:38.333580 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:38.333858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:38.334128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:38.336398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:38.398796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:38.400426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44534d3a-6673-4776-8f35-65c9de0db602 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:38.409827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1189/2413] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:38.510248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:38.512169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:38.525361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a61208-bc9f-4aa1-bff1-5d592fc01d69 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:38.529664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:38.530338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:38.530622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:38.531468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:38.532938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:38.614010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:38.615468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:38.624895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1225/2414] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:09:40.356102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:40.356622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:40.356733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:40.390995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:09:40.390995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:40.397915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1190/2415] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:09:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:09:40.411877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:40.425608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4243-7e02-4ad5-b626-800b3930e533 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:40.432707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:40.433670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:40.434053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:40.434424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:40.443953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:40.445605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82e25f-b717-4862-86c2-ce4f81072c2e service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:40.454333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1226/2416] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:09:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:09:50.373346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=31 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:50.450441 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=34 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:09:55.944103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:09:55.954365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c258d-6593-4900-8bb9-9abebfb99c41 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:09:55.958185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:09:55.958829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:09:55.959106 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:09:55.959374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:09:55.961209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:09:55.971991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:09:55.974074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:09:55.983011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1191/2417] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:09:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:00.423782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:00.424376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:00.424376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:00.447947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] MFA Rules not processed for user `2a54e9db924c440da63c04052fe058e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:10:00.449188 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:00.457906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1227/2418] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:10:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:10:05.624591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:10:05.626166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:05.642398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533e808-25a4-43f4-96c6-58aca722cf15 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:10:05.648440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:10:05.649125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:05.649408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:05.649683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:05.651498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:05.712373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:10:05.714054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:10:05.724128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1192/2419] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:10:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:06.632520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:10:06.652783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc138951-f4b4-4684-86e7-90fc91eb0306 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:10:06.658539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:10:06.659206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:06.659707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:06.660184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:06.662353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:06.676200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:10:06.678129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9e133-ea5f-47ed-a233-476a9063e7d2 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:10:06.690860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1228/2420] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:10:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:15.719716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=34 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:10:16.685677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=9 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:10:49.194911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:10:49.207138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb96d-3af2-4cd3-9561-4915a47ed50e service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:10:49.213912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:10:49.214638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:49.214923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:49.215282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:49.217286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:49.284368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:10:49.286244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:10:49.296303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1193/2421] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:10:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:55.508931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:55.509499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:55.509499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:55.530419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] MFA Rules not processed for user `9f56095d6bac4f108a73790ac9004f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:10:55.531902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:55.540064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1229/2422] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:10:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:10:56.026914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:10:56.037341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a54ef-e8a5-4761-86cb-934a9a38f537 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:10:56.044080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:10:56.045140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:56.045660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:56.046115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:56.048175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:56.062210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:10:56.064041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc02e95-a230-4d58-864f-bdddf5d87299 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:10:56.073654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1194/2423] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:10:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:56.570116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:10:56.571672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:56.662750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e9b1-f70e-4742-8526-8f0e90817571 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:10:56.668447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:10:56.669160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:56.669436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:56.669712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:56.671563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:56.684364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:10:56.686036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:10:56.697603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1230/2424] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:10:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:10:59.952649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:10:59.953266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:10:59.953266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:10:59.971949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] MFA Rules not processed for user `702dcd49668c4680b5bf3c1c75575ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:10:59.973212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:10:59.980540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1195/2425] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:06.692617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=35 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:08.824728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:08.839131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34387328-1af2-4b6c-8ea3-1e8d2911002f service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:08.847713 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:08.848497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:08.848841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:08.849194 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:08.851160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:08.877287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:08.879009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fea18b-1818-4148-89e5-8f6b17c8be79 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:08.893283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1231/2426] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:09.967121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=37 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:13.829463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:13.852213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fc28b-b8ce-442d-82d2-d30b673a5a65 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:13.859113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:13.860172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:13.860650 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:13.861153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:13.863946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:13.888746 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:13.892892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:13.907959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1196/2427] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:18.887971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:23.902498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:24.596178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:24.596689 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:24.596774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:24.617598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:24.618994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:24.626428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1232/2428] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:24.678749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:24.692071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bf2d-4c45-46b5-bef2-872a7731efa1 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:24.698487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:24.699155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:24.699474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:24.700089 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:24.714772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:24.717107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:24.726759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1197/2429] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:11:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:11:28.437528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:28.453566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb34378-a301-4fa4-b053-51c9664ea471 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:28.469414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:28.470190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:28.470573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:28.470851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:28.473426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:28.490236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:28.492394 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:28.503719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1233/2430] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:32.330481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:32.332387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:32.406135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e982a-eb71-4d0d-a55a-8cd70ca5b7ce service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:32.410571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:32.411452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:32.412032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:32.412434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] PATH_INFO: `/v3/users/a9728aa79bb04cf58ea266102a3a5119` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:32.413657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9728aa79bb04cf58ea266102a3a5119)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:32.415631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:32.466240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9728aa79bb04cf58ea266102a3a5119'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.473237 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9728aa79bb04cf58ea266102a3a5119'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.482412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9728aa79bb04cf58ea266102a3a5119'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.483032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9728aa79bb04cf58ea266102a3a5119'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.483756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9728aa79bb04cf58ea266102a3a5119'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.501540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1198/2431] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:32 2022] DELETE /identity/v3/users/a9728aa79bb04cf58ea266102a3a5119 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:32.680331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:32.693164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e95df-1c77-4299-800e-5937fb2616ad service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:32.753235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:32.754146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:32.754426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:32.754702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] PATH_INFO: `/v3/projects/734fc82eeb014ad7b7bdb1c09236d44b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:32.755917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734fc82eeb014ad7b7bdb1c09236d44b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:32.757878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:32.810746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734fc82eeb014ad7b7bdb1c09236d44b', 'request_id': 'req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.817232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734fc82eeb014ad7b7bdb1c09236d44b', 'request_id': 'req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.823130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734fc82eeb014ad7b7bdb1c09236d44b', 'request_id': 'req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.837710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] The token cache is being invalidate because project 734fc82eeb014ad7b7bdb1c09236d44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:32.838040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.838655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.840230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1234/2432] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:32 2022] DELETE /identity/v3/projects/734fc82eeb014ad7b7bdb1c09236d44b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:32.844503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:32.845950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:32.859173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e317457-a2f2-4fff-bc5f-920a120c9861 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:32.922240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:32.923250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:32.923551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:32.923829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] PATH_INFO: `/v3/users/3101c00ce7f54f5a8e4fcc462158bc0b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:32.925127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3101c00ce7f54f5a8e4fcc462158bc0b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:32.927049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:32.932754 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:32.976820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3101c00ce7f54f5a8e4fcc462158bc0b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.984127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3101c00ce7f54f5a8e4fcc462158bc0b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.993203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3101c00ce7f54f5a8e4fcc462158bc0b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.993853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3101c00ce7f54f5a8e4fcc462158bc0b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:32.994709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3101c00ce7f54f5a8e4fcc462158bc0b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.012681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1199/2433] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:32 2022] DELETE /identity/v3/users/3101c00ce7f54f5a8e4fcc462158bc0b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:33.190962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:33.192674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:33.207056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4d3f-c47d-421b-8f9e-80801a2904f3 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:33.268479 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:33.269383 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:33.269661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:33.269936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] PATH_INFO: `/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:33.271125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe1e6834c2f4581b3ce7a50dcada420)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:33.273091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:33.284227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:33.319565 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe1e6834c2f4581b3ce7a50dcada420', 'request_id': 'req-afa28aa8-12dd-4da2-a415-534959bb0372'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.325693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe1e6834c2f4581b3ce7a50dcada420', 'request_id': 'req-afa28aa8-12dd-4da2-a415-534959bb0372'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.331262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe1e6834c2f4581b3ce7a50dcada420', 'request_id': 'req-afa28aa8-12dd-4da2-a415-534959bb0372'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.344771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] The token cache is being invalidate because project 0fe1e6834c2f4581b3ce7a50dcada420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:33.345099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.345756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:33.347412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1235/2434] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:33 2022] DELETE /identity/v3/projects/0fe1e6834c2f4581b3ce7a50dcada420 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:37.056863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:37.058518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:37.120214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eb003-6b86-4a0a-9caf-3ca8eeea8021 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:37.172811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:37.173471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:37.173745 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:37.174013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:37.175855 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:37.231659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:37.245085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:37.246660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:37.257444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1200/2435] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:40.562890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:40.564424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:40.578621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa28aa8-12dd-4da2-a415-534959bb0372 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:40.587157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:40.589062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:40.589349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:40.589631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] PATH_INFO: `/v3/users/2a54e9db924c440da63c04052fe058e6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:40.591766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a54e9db924c440da63c04052fe058e6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:40.594053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:40.659971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a54e9db924c440da63c04052fe058e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:40.667814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a54e9db924c440da63c04052fe058e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:40.678102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a54e9db924c440da63c04052fe058e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:40.678791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a54e9db924c440da63c04052fe058e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:40.679407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7927ff1b-0e28-4eda-87b0-85729e714603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a54e9db924c440da63c04052fe058e6'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:40.698204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1236/2436] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:40 2022] DELETE /identity/v3/users/2a54e9db924c440da63c04052fe058e6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:11:40.938878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:40.956442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddb7fa-2a67-4180-93cc-e3bb700caaec service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.037451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.038394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.038676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.038947 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] PATH_INFO: `/v3/projects/c47f35f9ae5e4a55be23a84e3887b915` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.040574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47f35f9ae5e4a55be23a84e3887b915)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.042502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.116433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47f35f9ae5e4a55be23a84e3887b915', 'request_id': 'req-1d20931f-4680-4736-9592-02b1092fac40'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:41.126397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47f35f9ae5e4a55be23a84e3887b915', 'request_id': 'req-1d20931f-4680-4736-9592-02b1092fac40'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:41.133430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47f35f9ae5e4a55be23a84e3887b915', 'request_id': 'req-1d20931f-4680-4736-9592-02b1092fac40'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:41.154815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] The token cache is being invalidate because project c47f35f9ae5e4a55be23a84e3887b915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:41.155232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:41.156332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-1d20931f-4680-4736-9592-02b1092fac40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:41.158462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1201/2437] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:40 2022] DELETE /identity/v3/projects/c47f35f9ae5e4a55be23a84e3887b915 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:41.168369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.168369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.168369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.186141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:41.187388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:41.248651 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1237/2438] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:41.256396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.256684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.256989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.276084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:41.277401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:41.285666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1202/2439] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:41.290021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.300501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549a863-1832-4fa7-9cac-56cf6253b08e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.305947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.306643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.306926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.307201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.307947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:11:41.308475 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.310516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.319826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:41.327417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1238/2440] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:11:41.329085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.339116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb100c1d-1c14-4d7a-942f-1e305ee9659a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.344656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.345373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.345698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.346097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.346799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.348797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.390220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1203/2441] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:11:41 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:41.394335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.410414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3befcd-e01b-451d-aaa3-07560e6a67a0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.484898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.485631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.485919 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.486197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.487522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.489516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.517929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1239/2442] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:11:41 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:41.522433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.544343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628be1-014a-4794-ba64-3178faffa556 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.549547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.550271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.550520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.550797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.551695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.553386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.564760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1204/2443] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:41.568891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.579559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19e890-b9ad-4865-a2ab-a0be3594330f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.585048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.585724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.586004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.586274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.587003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.588616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9ae7e-11ea-4611-8d45-2042f90db67a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.598976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1240/2444] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:41.603184 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.613299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98f32c-77f7-4c1b-b5be-5b80e9922454 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.617863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.618757 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.619035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.619305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] PATH_INFO: `/v3/projects/96a57587a6d74100bf830d9fa3807df1/users/ecb66837ff6d450d8eb7f95d1e4bff32/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.635092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a57587a6d74100bf830d9fa3807df1, user_id=ecb66837ff6d450d8eb7f95d1e4bff32, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.637851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.652958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1205/2445] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:41 2022] PUT /identity/v3/projects/96a57587a6d74100bf830d9fa3807df1/users/ecb66837ff6d450d8eb7f95d1e4bff32/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:41.664272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.664488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.664764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.696367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] MFA Rules not processed for user `ecb66837ff6d450d8eb7f95d1e4bff32`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:41.697630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:41.763765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1241/2446] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:41.775513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:41.777346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:41.790074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:41.858054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426b7ae-0eea-41ae-9084-70581e8d04d9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:41.863772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:41.864436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:41.864726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:41.865028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:41.874208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:41.875837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca02ca-2a34-43dc-92d3-345648110a50 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:41.885358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1206/2447] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:42.107026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:42.169840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef559a-a60b-458e-b105-6dac6ad574f7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:42.175027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:42.176115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:42.176515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:42.176982 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] PATH_INFO: `/v3/users/ecb66837ff6d450d8eb7f95d1e4bff32` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:42.178299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb66837ff6d450d8eb7f95d1e4bff32)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:42.180352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:42.196395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:42.244534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb66837ff6d450d8eb7f95d1e4bff32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:42.259599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb66837ff6d450d8eb7f95d1e4bff32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:42.266748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb66837ff6d450d8eb7f95d1e4bff32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:42.267413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb66837ff6d450d8eb7f95d1e4bff32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:42.268004 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb66837ff6d450d8eb7f95d1e4bff32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:42.285627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1242/2448] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:42 2022] DELETE /identity/v3/users/ecb66837ff6d450d8eb7f95d1e4bff32 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:42.290501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:42.291024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:42.291124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:42.308156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:42.309409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:42.372224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1207/2449] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:11:42.383087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:42.384679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:42.446974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af5afe-8172-48f9-aec7-b4ff2b8bd0b1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:42.450501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:42.451732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:42.452201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:42.452373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:42.462667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:42.464102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707cf3-1cbe-4f40-a4ef-a6a778544e84 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:42.474720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1243/2450] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:42.926637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:42.928158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:42.955318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaf7b6-23c2-4f5a-afb9-08c5cd71ffac None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:42.962136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:42.963129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:42.963323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:42.963606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] PATH_INFO: `/v3/projects/96a57587a6d74100bf830d9fa3807df1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:42.964920 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a57587a6d74100bf830d9fa3807df1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:42.967887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.022311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a57587a6d74100bf830d9fa3807df1', 'request_id': 'req-7f9a8894-0ba6-4100-ac43-612ff8d39435'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:43.029596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a57587a6d74100bf830d9fa3807df1', 'request_id': 'req-7f9a8894-0ba6-4100-ac43-612ff8d39435'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:43.035531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a57587a6d74100bf830d9fa3807df1', 'request_id': 'req-7f9a8894-0ba6-4100-ac43-612ff8d39435'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:43.048530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] The token cache is being invalidate because project 96a57587a6d74100bf830d9fa3807df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:43.048896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:43.049553 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-7f9a8894-0ba6-4100-ac43-612ff8d39435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:43.051374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1208/2451] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:42 2022] DELETE /identity/v3/projects/96a57587a6d74100bf830d9fa3807df1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:43.056150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.056637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.056721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.076402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:43.077659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:43.145487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1244/2452] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:43.158676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.158962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.159241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.179842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:43.181073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:43.189475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1209/2453] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:43.193802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.204191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ff7e-7127-47cd-9328-d6b393aa6150 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.209057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.210102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.210368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.210639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.211371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:11:43.211672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.213314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.226547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1245/2454] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:11:43.230976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.240674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fc508-35ad-42c9-8b90-b0235669e030 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.246050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.246815 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.247073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.247644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.249493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.251607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.260659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:43.284945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1210/2455] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:11:43 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:43.288750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.299155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd1efa-9173-41d9-9da9-3fe912e8b31f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.313330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:43.362569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.363271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.363567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.363805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.364485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.366571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.390521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1246/2456] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:11:43 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:43.392031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.404084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ca98f-b4fe-4ef8-aa1a-e0bd37c38c29 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.412645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.413362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.413645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.413917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.414655 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.416327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.427363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1211/2457] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:43.431401 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.444487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c593af-e76e-41f6-89ad-f378850f1d17 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.449559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.450729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.451064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.451447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.452191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.453857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ab616-765d-4ca0-a76b-faef704f959c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.464781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1247/2458] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:43.469384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.479905 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6751-00b3-4df3-81d9-847cce8f6088 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.484616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.485535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.485811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.486080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] PATH_INFO: `/v3/projects/905b91a0ec94444a8f2ae5627dc13bc1/users/ce58fc2593074fba943da100ea0bf111/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.502452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905b91a0ec94444a8f2ae5627dc13bc1, user_id=ce58fc2593074fba943da100ea0bf111, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.505493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.526220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1212/2459] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:43 2022] PUT /identity/v3/projects/905b91a0ec94444a8f2ae5627dc13bc1/users/ce58fc2593074fba943da100ea0bf111/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:11:43.531944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b930e-1fd3-4f71-8060-35a3a5110223 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.532238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b930e-1fd3-4f71-8060-35a3a5110223 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.532515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b930e-1fd3-4f71-8060-35a3a5110223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.553214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f11b930e-1fd3-4f71-8060-35a3a5110223 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:43.554496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f11b930e-1fd3-4f71-8060-35a3a5110223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:43.618298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1248/2460] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:43.636174 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:43.638429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:43.699575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459fe17-d357-42bd-b167-5572cfe6d0b6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:43.703691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:43.704348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:43.704628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:43.704897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:43.714872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:43.716350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb52e7-8dee-46b5-83ad-0cb0871a8924 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:43.725900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1213/2461] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:44.400155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42adac-c939-4a65-ad08-651258640280 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.400672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42adac-c939-4a65-ad08-651258640280 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.400833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42adac-c939-4a65-ad08-651258640280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.420935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9d42adac-c939-4a65-ad08-651258640280 None None] MFA Rules not processed for user `b8d8d0c23f744a7b9ad311581f6ae39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:44.422171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9d42adac-c939-4a65-ad08-651258640280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:44.484318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1249/2462] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:44.647334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.647882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.647882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.679016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] MFA Rules not processed for user `ce58fc2593074fba943da100ea0bf111`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:44.680221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:44.757856 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1214/2463] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:44.758950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42adac-c939-4a65-ad08-651258640280 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:44.780213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42adac-c939-4a65-ad08-651258640280 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:44.792963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:44.793692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.793970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.794393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.795998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:44.798043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:44.827017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:44.828440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:44.830314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1250/2464] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:11:44 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:44.834360 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:44.845374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f5892-aa64-422c-a043-25af0eac5ac8 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:44.859258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:44.907660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:44.913236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb0a2b-395e-4e83-9c1b-1a094b1459a6 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:44.914098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:44.914832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.915058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.915336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.917672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:44.918192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:44.918860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.919216 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.920030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.920728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:44.922706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:44.927315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:44.928750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:44.937535 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1215/2465] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:11:44.944716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1251/2466] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:11:44 2022] POST /identity/v3/users => generated 309 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:44.948518 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:44.960922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971afa8d-08da-4961-a0ba-6d099b98ec19 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:44.965982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:44.966654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:44.966938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:44.967213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:44.967938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:44.969588 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:44.980303 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1216/2467] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:44.984875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:44.995406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868f9d-149c-4edf-b19d-382e8652f198 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:45.001292 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:45.002310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:45.002581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:45.002859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] PATH_INFO: `/v3/projects/fbf14c461ac1477890b4dbf235a1339d/users/6b2ee3b43f7c4a51ac393e8d580fb6cc/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:45.019147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf14c461ac1477890b4dbf235a1339d, user_id=6b2ee3b43f7c4a51ac393e8d580fb6cc, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:45.021892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:45.040190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1252/2468] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:44 2022] PUT /identity/v3/projects/fbf14c461ac1477890b4dbf235a1339d/users/6b2ee3b43f7c4a51ac393e8d580fb6cc/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:45.041153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:45.051459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbc8cb-1308-4957-bfab-afab97313ca1 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:45.111928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:45.112611 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:45.112892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:45.113194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:45.113921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:45.115545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa61545-8f02-4f47-9826-538eed595134 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:45.129776 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1217/2469] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:11:45.131145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:45.142852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2b47-a55e-4fcb-aa10-cc185babdf1b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:45.148921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:45.149868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:45.150229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:45.150398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] PATH_INFO: `/v3/projects/fbf14c461ac1477890b4dbf235a1339d/users/6b2ee3b43f7c4a51ac393e8d580fb6cc/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:45.152559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf14c461ac1477890b4dbf235a1339d, user_id=6b2ee3b43f7c4a51ac393e8d580fb6cc, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:45.156183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:45.169080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1253/2470] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:45 2022] PUT /identity/v3/projects/fbf14c461ac1477890b4dbf235a1339d/users/6b2ee3b43f7c4a51ac393e8d580fb6cc/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:45.881842 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:45.882380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:45.882380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:45.910630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] MFA Rules not processed for user `6b2ee3b43f7c4a51ac393e8d580fb6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:45.911906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:45.990764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1218/2471] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:47.348814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:47.431344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a6fc5-d116-4835-810e-5c455ba08cab admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:47.436150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:47.437089 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:47.437362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:47.437650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] PATH_INFO: `/v3/users/ce58fc2593074fba943da100ea0bf111` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:47.438974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce58fc2593074fba943da100ea0bf111)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:47.440955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:47.506355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce58fc2593074fba943da100ea0bf111'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.517371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce58fc2593074fba943da100ea0bf111'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.531508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce58fc2593074fba943da100ea0bf111'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.532313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce58fc2593074fba943da100ea0bf111'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.533022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce58fc2593074fba943da100ea0bf111'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.545412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:47.558706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1254/2472] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:47 2022] DELETE /identity/v3/users/ce58fc2593074fba943da100ea0bf111 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:47.764688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:47.785281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e41ba5-3bd5-4eb3-90e4-9778675c240d None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:47.864643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87091087-2596-471d-a253-28a5df798077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:47.865771 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091087-2596-471d-a253-28a5df798077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:47.866115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091087-2596-471d-a253-28a5df798077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:47.866472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091087-2596-471d-a253-28a5df798077 admin admin] PATH_INFO: `/v3/projects/905b91a0ec94444a8f2ae5627dc13bc1` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:47.867797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87091087-2596-471d-a253-28a5df798077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905b91a0ec94444a8f2ae5627dc13bc1)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:47.869851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87091087-2596-471d-a253-28a5df798077 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:47.871679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:47.924310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905b91a0ec94444a8f2ae5627dc13bc1', 'request_id': 'req-87091087-2596-471d-a253-28a5df798077'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.931462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905b91a0ec94444a8f2ae5627dc13bc1', 'request_id': 'req-87091087-2596-471d-a253-28a5df798077'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.937949 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905b91a0ec94444a8f2ae5627dc13bc1', 'request_id': 'req-87091087-2596-471d-a253-28a5df798077'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.951858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] The token cache is being invalidate because project 905b91a0ec94444a8f2ae5627dc13bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:47.952144 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.952872 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:47.954567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1219/2473] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:47 2022] DELETE /identity/v3/projects/905b91a0ec94444a8f2ae5627dc13bc1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:47.958970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:47.960746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:47.974591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbff20-7588-4f33-b719-ba5127dbfc5e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.040024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.041067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.041356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.041640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] PATH_INFO: `/v3/users/6b2ee3b43f7c4a51ac393e8d580fb6cc` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.042865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2ee3b43f7c4a51ac393e8d580fb6cc)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.044886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.110381 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2ee3b43f7c4a51ac393e8d580fb6cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.119677 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2ee3b43f7c4a51ac393e8d580fb6cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.129243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2ee3b43f7c4a51ac393e8d580fb6cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.129994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2ee3b43f7c4a51ac393e8d580fb6cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.130599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-6ad388b4-c7c8-4279-a5b3-343e8739a5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2ee3b43f7c4a51ac393e8d580fb6cc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.151397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1255/2474] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:11:47 2022] DELETE /identity/v3/users/6b2ee3b43f7c4a51ac393e8d580fb6cc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:48.383346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.385195 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:48.404614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87091087-2596-471d-a253-28a5df798077 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.468493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.469404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.469683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.469961 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] PATH_INFO: `/v3/projects/fbf14c461ac1477890b4dbf235a1339d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.471162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf14c461ac1477890b4dbf235a1339d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.473297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.520418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf14c461ac1477890b4dbf235a1339d', 'request_id': 'req-46fb8553-3f79-4700-bcbb-ead85c8420ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.526616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf14c461ac1477890b4dbf235a1339d', 'request_id': 'req-46fb8553-3f79-4700-bcbb-ead85c8420ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.532882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf14c461ac1477890b4dbf235a1339d', 'request_id': 'req-46fb8553-3f79-4700-bcbb-ead85c8420ec'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.547782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] The token cache is being invalidate because project fbf14c461ac1477890b4dbf235a1339d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:11:48.548085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.548768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-46fb8553-3f79-4700-bcbb-ead85c8420ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:11:48.550910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1220/2475] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:11:48 2022] DELETE /identity/v3/projects/fbf14c461ac1477890b4dbf235a1339d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:48.556062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.556544 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.556648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.575669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:48.577195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:48.637647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1256/2476] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:48.645804 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.646175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.646541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.666407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:48.667618 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:48.675980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1221/2477] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:48.680777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.692377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab4721-726e-4db4-b62f-38473d0469b3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.698228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.698930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.699218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.699499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.700382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:11:48.700698 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.702398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.710109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:48.715606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1257/2478] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:11:48.720228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.730353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ac85-6350-484f-9f6e-578518f8b1ed None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.736318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.737017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.737299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.737597 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.738281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.740219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.750656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:48.770329 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1222/2479] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:11:48 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:11:48.774246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.784808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a32b-dcac-4d66-8621-983f8bf9d844 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.847477 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.848160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.848440 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.848715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.849573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.851500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.874683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1258/2480] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:11:48 2022] POST /identity/v3/users => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:48.879231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.890950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4c53-049e-4fd3-8f41-3e69deaf217e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.897328 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.898022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.898298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.898566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.899297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.900942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.912315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1223/2481] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:48.916457 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.927753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8785b88-7052-41df-9bce-e293dab02e8b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.933721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.934382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.934726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.935013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.935744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.937402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22a7f6-90b0-4a27-90c2-b0e611f9f820 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:48.948337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1259/2482] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:48.952885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:48.963894 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfd92-4ccb-49a9-9ab3-23380ec4c9d3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:48.969224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:48.970116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:48.970397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:48.970680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] PATH_INFO: `/v3/projects/276ce8f07c6843dcbf824d9436b4e906/users/5e19bbd4874e4eb7bd88fad5b2942e8f/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:48.987235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=276ce8f07c6843dcbf824d9436b4e906, user_id=5e19bbd4874e4eb7bd88fad5b2942e8f, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:48.989970 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.004539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1224/2483] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:48 2022] PUT /identity/v3/projects/276ce8f07c6843dcbf824d9436b4e906/users/5e19bbd4874e4eb7bd88fad5b2942e8f/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:49.015094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.015389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.015679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.043707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] MFA Rules not processed for user `5e19bbd4874e4eb7bd88fad5b2942e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:49.045476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:49.118731 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1260/2484] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:49 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:49.123953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.188454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075f15b-5721-45a2-b201-4ff55fc2eefa admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.194406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.195093 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.195354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.195651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.196336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.198319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.226779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1225/2485] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:11:49 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:11:49.230986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.241971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8966-a32d-49f5-b467-69df92fec199 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.300551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.301187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.301469 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.301744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.302438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.304378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.314382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:49.327811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1261/2486] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:11:49 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:11:49.337506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.348462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae99229-f74e-46a3-bda2-10f96e1e9914 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.357430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.358416 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.358679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.359153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.360280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.362825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.377319 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1226/2487] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:49.381740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.392213 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e1c5b-c717-4ad6-956e-1fb55a226369 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.397034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.397921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.398198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.398476 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] PATH_INFO: `/v3/projects/88c309c57c6644c19ccfba9e7ca128b8/users/2bf4cc7b461b4083ab270fd6b091f206/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.414258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c309c57c6644c19ccfba9e7ca128b8, user_id=2bf4cc7b461b4083ab270fd6b091f206, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.416946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.432371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1262/2488] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:49 2022] PUT /identity/v3/projects/88c309c57c6644c19ccfba9e7ca128b8/users/2bf4cc7b461b4083ab270fd6b091f206/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:49.436865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.451858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bc754-47d2-4c34-8fea-7691047ec75f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.511701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.512376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.512661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.513323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.514057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.515677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e333c49-9e61-424e-9e75-827717bd38c1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.525801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1227/2489] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:11:49.530557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.541879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15c664-5b66-4dae-bfb1-e95409c261ce admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.547464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.548361 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.548705 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.549095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] PATH_INFO: `/v3/projects/88c309c57c6644c19ccfba9e7ca128b8/users/2bf4cc7b461b4083ab270fd6b091f206/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.551646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c309c57c6644c19ccfba9e7ca128b8, user_id=2bf4cc7b461b4083ab270fd6b091f206, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.554511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.567279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1263/2490] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:11:49 2022] PUT /identity/v3/projects/88c309c57c6644c19ccfba9e7ca128b8/users/2bf4cc7b461b4083ab270fd6b091f206/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:11:49.580141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.580586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.581032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.611541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] MFA Rules not processed for user `2bf4cc7b461b4083ab270fd6b091f206`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:11:49.612929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:49.627391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:49.685338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1228/2491] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:11:49 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:11:49.697329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.699791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:11:49.761606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039cfa-709d-4f42-9200-cab7236a2719 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.765959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.766620 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.766903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.767173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.777571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.779183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.788769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1264/2492] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:49 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:49.869788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:11:49.880113 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8661e5d-0634-4a23-b44f-803a238c0dc5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:11:49.884062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:11:49.884742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:11:49.885057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:11:49.885331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:11:49.949368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:11:49.950935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be9ec3-f21e-48b9-a206-f88b59503ddf service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:11:49.964549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1229/2493] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:11:49 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:11:59.784456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:11:59.957719 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=50 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:04.982210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:05.046154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca83b7-611d-4e7b-b4b2-3797b7225097 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:05.054038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:05.055345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:05.055717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:05.056095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] PATH_INFO: `/v3/users/5e19bbd4874e4eb7bd88fad5b2942e8f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:05.057778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e19bbd4874e4eb7bd88fad5b2942e8f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:05.060790 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:05.111272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e19bbd4874e4eb7bd88fad5b2942e8f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.119358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e19bbd4874e4eb7bd88fad5b2942e8f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.126864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e19bbd4874e4eb7bd88fad5b2942e8f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.127491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e19bbd4874e4eb7bd88fad5b2942e8f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.128183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e19bbd4874e4eb7bd88fad5b2942e8f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.145839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1265/2494] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:04 2022] DELETE /identity/v3/users/5e19bbd4874e4eb7bd88fad5b2942e8f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:05.150792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:05.151315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:05.151427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:05.175242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:05.176468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:05.238267 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1230/2495] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:12:05.248253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:05.249751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:05.309142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4f6b6-3130-4f32-ba35-e69a1a35923e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:05.313965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:05.314543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:05.314819 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:05.315117 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:05.328336 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:05.329848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:05.339958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1266/2496] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:05.637552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:05.639136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:05.651687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2b29-95d1-47b7-93fd-7b2a867a8219 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:05.657091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:05.658073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:05.658307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:05.658617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] PATH_INFO: `/v3/projects/276ce8f07c6843dcbf824d9436b4e906` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:05.660011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=276ce8f07c6843dcbf824d9436b4e906)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:05.662582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:05.713226 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276ce8f07c6843dcbf824d9436b4e906', 'request_id': 'req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.719345 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276ce8f07c6843dcbf824d9436b4e906', 'request_id': 'req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.725071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276ce8f07c6843dcbf824d9436b4e906', 'request_id': 'req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.738418 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] The token cache is being invalidate because project 276ce8f07c6843dcbf824d9436b4e906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:05.738706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.739402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.741145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1231/2497] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:05 2022] DELETE /identity/v3/projects/276ce8f07c6843dcbf824d9436b4e906 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:05.745538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:05.755590 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca779d9-cff6-4af9-b1f8-75aeb5b2f2a0 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:05.776352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:05.820135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:05.821067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:05.821355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:05.821635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] PATH_INFO: `/v3/users/2bf4cc7b461b4083ab270fd6b091f206` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:05.822850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf4cc7b461b4083ab270fd6b091f206)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:05.824808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:05.879718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf4cc7b461b4083ab270fd6b091f206'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.888995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf4cc7b461b4083ab270fd6b091f206'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.896846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf4cc7b461b4083ab270fd6b091f206'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.897506 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf4cc7b461b4083ab270fd6b091f206'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.898120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-da7a758c-49bb-485b-8f98-4472773c3448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf4cc7b461b4083ab270fd6b091f206'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:05.915139 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1267/2498] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:05 2022] DELETE /identity/v3/users/2bf4cc7b461b4083ab270fd6b091f206 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:12:06.232813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.234338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:06.249673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb1b64-7b6c-4d74-aaa2-19f5532e25d4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.309096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.309969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.310247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.310522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] PATH_INFO: `/v3/projects/88c309c57c6644c19ccfba9e7ca128b8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.311704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c309c57c6644c19ccfba9e7ca128b8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.313748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.348867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:06.366962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c309c57c6644c19ccfba9e7ca128b8', 'request_id': 'req-5c4006de-05bc-43f7-a282-2e90cfa22d29'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:06.373002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c309c57c6644c19ccfba9e7ca128b8', 'request_id': 'req-5c4006de-05bc-43f7-a282-2e90cfa22d29'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:06.379403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c309c57c6644c19ccfba9e7ca128b8', 'request_id': 'req-5c4006de-05bc-43f7-a282-2e90cfa22d29'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:06.392864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] The token cache is being invalidate because project 88c309c57c6644c19ccfba9e7ca128b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:06.393227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:06.393884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c4006de-05bc-43f7-a282-2e90cfa22d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:06.395482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1232/2499] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:06 2022] DELETE /identity/v3/projects/88c309c57c6644c19ccfba9e7ca128b8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:12:06.400521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.401033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.401146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.419898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:06.421146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:06.492462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1268/2500] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:12:06.501230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.501514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.501792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.525039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:06.526256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:06.536887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1233/2501] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:06.541931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.555549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eaa71f-37bc-464a-9142-486330a45354 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.561027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.561741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.562022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.562299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.563241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:12:06.563567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.565226 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.577802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1269/2502] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:12:06.582470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.592304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7f05-6d04-4eb4-bf07-1d8cc003d494 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.597583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.598243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.598506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.598772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.599456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.601404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.628504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1234/2503] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:06.632753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.643356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bec3e-53c1-4dbe-b9cc-de6fa482ce4c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.670355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:06.702458 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.703233 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.703547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.703846 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.704564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.706537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.728814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1270/2504] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:06.733095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.743735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76df92-da30-4a4b-a807-1fd95ac70d83 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.748596 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.749275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.749549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.749817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.750955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.752558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.763065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1235/2505] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:06.767206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.777423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289a84d-9ac7-4632-99bb-14730e1f2d16 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.783434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.784097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.784520 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.784784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.785568 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.787179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8d8a3-ad0f-4115-8d8b-b809b848c4ce admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.798875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1271/2506] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:06.804150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:06.816014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9292b9-7e7a-4039-a02d-196737562548 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:06.822047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:06.822964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.823244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.823516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] PATH_INFO: `/v3/projects/5dbcff4ed7cb41f990c10020402be9e2/users/78c57b14b5cd4802ba143fe3e0d34a11/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.840148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbcff4ed7cb41f990c10020402be9e2, user_id=78c57b14b5cd4802ba143fe3e0d34a11, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:06.842925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:06.863955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1236/2507] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:06 2022] PUT /identity/v3/projects/5dbcff4ed7cb41f990c10020402be9e2/users/78c57b14b5cd4802ba143fe3e0d34a11/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:06.878044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:06.878328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:06.878600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:06.907271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] MFA Rules not processed for user `78c57b14b5cd4802ba143fe3e0d34a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:06.908468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:06.977573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1272/2508] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:06.982281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.047159 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669ea9e-daf5-4e1b-8ef1-cc6f9b3718a6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.052430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.053120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.053396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.053656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.054343 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.056274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.083357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1237/2509] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:12:06 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:07.088308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.099325 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e9b59-84f3-4f5b-a808-aefdc59b757d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.165880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.166478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.166760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.167035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.167719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.169944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.192439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1273/2510] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:12:07 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:07.196684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.207718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6981-2c4d-451a-8e95-1160c3534c84 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.213182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.213886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.214241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.214515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.215364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.217165 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.228870 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1238/2511] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:07.234024 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.251507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c0ee-b29b-4b41-a7cf-21858cb8e074 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.257505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.258392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.258670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.258938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] PATH_INFO: `/v3/projects/95c8547594fa4001801d2f653582ce0c/users/845ed71233f345adbabc4d2447ef5e09/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.274906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c8547594fa4001801d2f653582ce0c, user_id=845ed71233f345adbabc4d2447ef5e09, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.277631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.292724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1274/2512] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:07 2022] PUT /identity/v3/projects/95c8547594fa4001801d2f653582ce0c/users/845ed71233f345adbabc4d2447ef5e09/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:07.297063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.307680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f7ad8-63b4-4847-b315-4817d4f9fa7d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.327011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:07.368456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.369164 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.369447 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.369718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.370452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.372088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863afe9-634c-4ceb-9bb4-1a130129b6f1 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.387263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1239/2513] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:07.391750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.397339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:07.402850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0395a-080c-4e3f-8eab-b25bd31a1560 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.408589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.409529 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.409807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.410063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] PATH_INFO: `/v3/projects/95c8547594fa4001801d2f653582ce0c/users/845ed71233f345adbabc4d2447ef5e09/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.412054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c8547594fa4001801d2f653582ce0c, user_id=845ed71233f345adbabc4d2447ef5e09, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.414751 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.425319 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1275/2514] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:07 2022] PUT /identity/v3/projects/95c8547594fa4001801d2f653582ce0c/users/845ed71233f345adbabc4d2447ef5e09/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:07.435461 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.435751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.436049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.464135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] MFA Rules not processed for user `845ed71233f345adbabc4d2447ef5e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:07.465362 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:07.547003 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1240/2515] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:07.560496 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.563683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:07.627461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0bae9-f510-4cc4-a177-d5845e818db2 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.631351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.632000 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.632276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.632545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.641900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.643333 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.652625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1276/2516] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:07.744690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:07.755718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ed1f7-5f41-4292-b4ec-7689610f879c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:07.759781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:07.760400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:07.760675 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:07.760983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:07.822452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:07.823943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924d7bf-b73e-44da-afd3-d58dba9ad358 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:07.835278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1241/2517] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:08.127770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:08.197774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274bbe-46ad-474f-b15a-c04123e886e5 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:08.205236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:08.206130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:08.206414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:08.206685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] PATH_INFO: `/v3/users/78c57b14b5cd4802ba143fe3e0d34a11` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:08.207992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c57b14b5cd4802ba143fe3e0d34a11)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:08.210366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:08.266644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c57b14b5cd4802ba143fe3e0d34a11'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.275746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c57b14b5cd4802ba143fe3e0d34a11'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.283805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c57b14b5cd4802ba143fe3e0d34a11'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.284554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c57b14b5cd4802ba143fe3e0d34a11'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.285265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c57b14b5cd4802ba143fe3e0d34a11'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.303310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1277/2518] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:08 2022] DELETE /identity/v3/users/78c57b14b5cd4802ba143fe3e0d34a11 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:08.307906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:08.308376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:08.308469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:08.326330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:08.327552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:08.370549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:08.391552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1242/2519] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:08.404493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:08.406090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:08.424688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:08.475491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f7f1-62a8-4ee1-a81b-a1ca55d763ac admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:08.479206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:08.479862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:08.480144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:08.480418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:08.489703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:08.491151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:08.500822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1278/2520] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:08.787756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:08.789698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:08.802625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfef4e4-ee56-4cd3-8049-2a5c1573a929 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:08.808354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:08.809295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:08.809569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:08.809828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] PATH_INFO: `/v3/projects/5dbcff4ed7cb41f990c10020402be9e2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:08.811145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbcff4ed7cb41f990c10020402be9e2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:08.813325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:08.869028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbcff4ed7cb41f990c10020402be9e2', 'request_id': 'req-d445828d-3e89-41d4-9f02-35de4f152cde'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.876173 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbcff4ed7cb41f990c10020402be9e2', 'request_id': 'req-d445828d-3e89-41d4-9f02-35de4f152cde'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.882838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbcff4ed7cb41f990c10020402be9e2', 'request_id': 'req-d445828d-3e89-41d4-9f02-35de4f152cde'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.899054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] The token cache is being invalidate because project 5dbcff4ed7cb41f990c10020402be9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:08.899342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.900101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:08.901742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1243/2521] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:08 2022] DELETE /identity/v3/projects/5dbcff4ed7cb41f990c10020402be9e2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:08.905993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:08.917190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcf506-c29b-4e4b-847e-e7115e283877 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:08.980369 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:08.981306 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:08.981588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:08.981867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] PATH_INFO: `/v3/users/845ed71233f345adbabc4d2447ef5e09` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:08.983122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=845ed71233f345adbabc4d2447ef5e09)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:08.985157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.035906 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845ed71233f345adbabc4d2447ef5e09'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.044787 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845ed71233f345adbabc4d2447ef5e09'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.052123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845ed71233f345adbabc4d2447ef5e09'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.052753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845ed71233f345adbabc4d2447ef5e09'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.054107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-65bc36a8-7396-4622-bae1-781943ddfed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845ed71233f345adbabc4d2447ef5e09'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.071637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1279/2522] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:08 2022] DELETE /identity/v3/users/845ed71233f345adbabc4d2447ef5e09 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:09.346831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.348410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:09.361562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445828d-3e89-41d4-9f02-35de4f152cde admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.421896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.422761 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.423045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.423320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] PATH_INFO: `/v3/projects/95c8547594fa4001801d2f653582ce0c` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.424505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c8547594fa4001801d2f653582ce0c)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.426483 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.475321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c8547594fa4001801d2f653582ce0c', 'request_id': 'req-ca708b31-f277-40b9-8bd2-26e802b6678b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.481525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c8547594fa4001801d2f653582ce0c', 'request_id': 'req-ca708b31-f277-40b9-8bd2-26e802b6678b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.487357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c8547594fa4001801d2f653582ce0c', 'request_id': 'req-ca708b31-f277-40b9-8bd2-26e802b6678b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.497400 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:09.502378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] The token cache is being invalidate because project 95c8547594fa4001801d2f653582ce0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:09.502668 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.503380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ca708b31-f277-40b9-8bd2-26e802b6678b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:09.505432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1244/2523] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:09 2022] DELETE /identity/v3/projects/95c8547594fa4001801d2f653582ce0c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:09.511036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.511533 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.511634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.531056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:09.532274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:09.564534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:09.598305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1280/2524] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:09.606579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.606860 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.607136 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.628893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:09.630155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:09.638933 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1245/2525] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:09.643330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.653737 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474f899-0f05-49fb-b864-a06ece2fa7cc None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.658652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.659346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.659635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.659927 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.660648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:12:09.660981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.662608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.674268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1281/2526] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:12:09.678621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.688987 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80165-632c-42c8-b8c1-504690951d13 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.694739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.695391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.695670 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.695936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.696616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.698612 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.731047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1246/2527] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:12:09 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:09.735295 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.746277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7d8f-d754-478f-b594-66984bf8b944 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.807083 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.807777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.808056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.808330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.809065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.811087 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.832808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1282/2528] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:12:09 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:09.837338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.850667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e392e2-f7e1-4565-9189-52d7a3f0da82 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.856958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.857942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.858317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.858678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.859782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.862411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.878810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1247/2529] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:09.883215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.894434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b481e0-5fb9-4fe9-92a9-1214164bbafd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.900554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.901248 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.901526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.901871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.902498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.904134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa8b24-a35b-4825-bdcd-6342fa0e5b12 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.915862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1283/2530] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:09.920280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:09.930854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936cb5-139e-43ba-afbc-bbd28272b512 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:09.935733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:09.936617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.936892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.937197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] PATH_INFO: `/v3/projects/79c1e791e04247db902ee04af29d0b14/users/2f96905c7bfa4e18bcee7050ae61fe91/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.951547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c1e791e04247db902ee04af29d0b14, user_id=2f96905c7bfa4e18bcee7050ae61fe91, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:09.954271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:09.969280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1248/2531] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:09 2022] PUT /identity/v3/projects/79c1e791e04247db902ee04af29d0b14/users/2f96905c7bfa4e18bcee7050ae61fe91/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:09.976549 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456eb62-f09c-4ba0-8083-2b793f2e790c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:09.976824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456eb62-f09c-4ba0-8083-2b793f2e790c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:09.977138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456eb62-f09c-4ba0-8083-2b793f2e790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:09.996097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e456eb62-f09c-4ba0-8083-2b793f2e790c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:09.997352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e456eb62-f09c-4ba0-8083-2b793f2e790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:10.062130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1284/2532] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:12:10.072218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:10.074208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:10.131661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f249b-d847-47e1-9076-57ec35329cd9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:10.135628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:10.136270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:10.136546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:10.136816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:10.149056 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:10.150514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:10.160273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1249/2533] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:12:10.806299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:10.806873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:10.806873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:10.838279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] MFA Rules not processed for user `2f96905c7bfa4e18bcee7050ae61fe91`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:10.841091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:10.871581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:10.914796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1285/2534] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:10.919610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:10.929824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add8722-ae88-423b-944f-576b5f0e9f67 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:10.934712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:10.935380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:10.935662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:10.935939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:10.936627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:10.938620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:10.966028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1250/2535] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:12:10 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:10.970608 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:10.982808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d7e23-05e8-4dc1-b3c8-90203bd3bf82 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:11.040228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:11.040931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.041221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.041500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.042204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:11.044144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:11.068069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1286/2536] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:12:10 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:12:11.073547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:11.089221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d758b-b858-4e94-9fc6-0692f52780a4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:11.094758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:11.095427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.095707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.096012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.096744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:11.099011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:11.109873 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1251/2537] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:11.115905 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:11.130472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1d0e9-04de-4937-b2ee-265565294fe7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:11.136820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:11.137748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.138030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.138302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] PATH_INFO: `/v3/projects/54e4a05de979482dbb4ba9805ffb653d/users/46139282db8c4e34877e8d589287ac34/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.155539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e4a05de979482dbb4ba9805ffb653d, user_id=46139282db8c4e34877e8d589287ac34, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:11.158312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:11.177091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1287/2538] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:11 2022] PUT /identity/v3/projects/54e4a05de979482dbb4ba9805ffb653d/users/46139282db8c4e34877e8d589287ac34/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:11.182955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:11.196895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90698ae-b6e6-40dd-877c-3ef1fd5d95f5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:11.255169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:11.255844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.256121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.256392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.257152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:11.258772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36100e3-67ba-4062-976c-84a4b61d4713 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:11.263853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:11.269215 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1252/2539] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:12:11.273658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:11.284314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaeda1-3150-42c6-8e11-012477d61dde admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:11.289385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:11.290285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.290567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.290841 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] PATH_INFO: `/v3/projects/54e4a05de979482dbb4ba9805ffb653d/users/46139282db8c4e34877e8d589287ac34/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.292974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e4a05de979482dbb4ba9805ffb653d, user_id=46139282db8c4e34877e8d589287ac34, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:11.295645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:11.306491 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1288/2540] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:11 2022] PUT /identity/v3/projects/54e4a05de979482dbb4ba9805ffb653d/users/46139282db8c4e34877e8d589287ac34/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:11.930082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:11.930605 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:11.930950 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:11.959572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] MFA Rules not processed for user `46139282db8c4e34877e8d589287ac34`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:11.960763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:12.035318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1253/2541] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:12.043423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:12.045376 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:12.107242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13caa-8e06-4147-b2f9-7a28bb3495d1 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:12.110904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:12.111562 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:12.111844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:12.112120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:12.147659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:12.171227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:12.172667 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1e290-eec9-4a6d-a6bc-e7987ee2fa60 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:12.181814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1289/2542] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:13.575419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:13.577127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:13.590248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8300c-2781-4bda-bf18-a390942ac782 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:13.594800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:13.595476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:13.595756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:13.596030 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:13.606901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:13.608361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f612e5-45ef-4902-a36e-26543cab22d2 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:13.618586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1254/2543] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:22.178235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=11 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:23.613885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=40 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:26.831567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019826-4283-49c4-91b7-42bab2d45abf None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:26.832178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019826-4283-49c4-91b7-42bab2d45abf None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:26.832178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71019826-4283-49c4-91b7-42bab2d45abf None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:26.834275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1290/2544] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:12:26.850332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:26.850863 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:26.850995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:26.888411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] MFA Rules not processed for user `9bb4ac455d464617840790998ceea9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:26.892028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:26.914091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71019826-4283-49c4-91b7-42bab2d45abf None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:26.962868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1255/2545] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:12:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:12:26.976638 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:26.978327 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:26.980701 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71019826-4283-49c4-91b7-42bab2d45abf None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:26.985338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:26.986225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:26.986613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:26.986862 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] PATH_INFO: `/v3/users/2f96905c7bfa4e18bcee7050ae61fe91` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:26.988115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f96905c7bfa4e18bcee7050ae61fe91)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:26.990096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:26.991076 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571ca56-886e-409c-b26b-e8b86d6cece3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:26.996214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:26.996854 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:26.997236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:26.997506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:27.006317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:27.007936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:27.018183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1256/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:12:27.039044 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f96905c7bfa4e18bcee7050ae61fe91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.047970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f96905c7bfa4e18bcee7050ae61fe91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.055473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f96905c7bfa4e18bcee7050ae61fe91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.056189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f96905c7bfa4e18bcee7050ae61fe91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.056957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f96905c7bfa4e18bcee7050ae61fe91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.074191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1291/2547] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:26 2022] DELETE /identity/v3/users/2f96905c7bfa4e18bcee7050ae61fe91 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:12:27.253709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:27.265819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d12774-a4cc-43f9-8a66-9dea7d0af9a2 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:27.327759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:27.328647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:27.328941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:27.329217 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] PATH_INFO: `/v3/projects/79c1e791e04247db902ee04af29d0b14` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:27.330520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c1e791e04247db902ee04af29d0b14)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:27.332460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:27.383703 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c1e791e04247db902ee04af29d0b14', 'request_id': 'req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.389848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c1e791e04247db902ee04af29d0b14', 'request_id': 'req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.395396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c1e791e04247db902ee04af29d0b14', 'request_id': 'req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.409549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:27.414029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] The token cache is being invalidate because project 79c1e791e04247db902ee04af29d0b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:27.414364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.415032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.417282 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1257/2548] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:27 2022] DELETE /identity/v3/projects/79c1e791e04247db902ee04af29d0b14 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:27.421765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:27.423109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:27.437653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3df36-487f-49f9-819c-252a02b7ae84 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:27.502007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:27.502898 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:27.503182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:27.503452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] PATH_INFO: `/v3/users/46139282db8c4e34877e8d589287ac34` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:27.504829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46139282db8c4e34877e8d589287ac34)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:27.506838 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:27.529873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:27.562500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46139282db8c4e34877e8d589287ac34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.574316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46139282db8c4e34877e8d589287ac34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.581462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46139282db8c4e34877e8d589287ac34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.582102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46139282db8c4e34877e8d589287ac34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.582714 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7917cdb3-e647-48ec-84f4-971fe93bdcf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46139282db8c4e34877e8d589287ac34'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.600844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1292/2549] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:12:27 2022] DELETE /identity/v3/users/46139282db8c4e34877e8d589287ac34 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:27.797868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:27.799452 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:27.812380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56efdf-f1ff-49e0-9509-ecbab1ecf72b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:27.876441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:27.877369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:27.877649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:27.877915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] PATH_INFO: `/v3/projects/54e4a05de979482dbb4ba9805ffb653d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:27.879104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e4a05de979482dbb4ba9805ffb653d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:27.881046 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:27.935419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e4a05de979482dbb4ba9805ffb653d', 'request_id': 'req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.944497 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e4a05de979482dbb4ba9805ffb653d', 'request_id': 'req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.952657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e4a05de979482dbb4ba9805ffb653d', 'request_id': 'req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.969948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] The token cache is being invalidate because project 54e4a05de979482dbb4ba9805ffb653d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:12:27.970251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.971128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a14a4680-82ef-4df5-bea5-ccc9c5ca084c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:12:27.972737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1258/2550] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:12:27 2022] DELETE /identity/v3/projects/54e4a05de979482dbb4ba9805ffb653d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:27.977778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:27.978272 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:27.978372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:27.997575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:27.998792 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:28.070566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1293/2551] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:28.079934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.080224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.080499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.099613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:28.100828 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:28.109100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1259/2552] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:28.113468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.123945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90044b6-0b2a-45c0-9dbc-477bb3841fa3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.130038 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.130693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.130974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.131262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.131969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:12:28.132273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.133921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.146480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1294/2553] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:12:28.151079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.162142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30307620-bb07-4c88-a2d7-46be8e98d256 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.168764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.169446 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.169732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.170005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.170697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.172637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.200706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1260/2554] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:12:28 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:12:28.204998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.215794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdd978-4f30-4439-91b5-743a2d5ba37a admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.274998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.275801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.276237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.276412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.277136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.279078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.300770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1295/2555] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:12:28 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:12:28.308478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.324075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da295015-5d8c-4f8f-9288-11142889d52a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.329734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.330406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.330686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.330954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.331689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.333364 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.345299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1261/2556] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:12:28.349702 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.359802 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ed1e4-cff8-44dd-8d7f-0d8aaca0f572 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.364517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.365211 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.365493 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.365767 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.366485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.368079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7667f8-5d6c-48ec-a018-1944de22079d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.378759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1296/2557] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:12:28.382900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.393514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905bd8-a5c1-47f1-82a0-cc91622c6771 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.398222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.399156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.399422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.399688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] PATH_INFO: `/v3/projects/2ac1aab84e994f7892964ff85819002d/users/ddb9e04ba5c046398965b7e79eb3280b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.414545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac1aab84e994f7892964ff85819002d, user_id=ddb9e04ba5c046398965b7e79eb3280b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.417382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.432925 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1262/2558] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:12:28 2022] PUT /identity/v3/projects/2ac1aab84e994f7892964ff85819002d/users/ddb9e04ba5c046398965b7e79eb3280b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:12:28.438926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01327-6a56-4d7a-a7a9-90b846bfb2b4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.439297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01327-6a56-4d7a-a7a9-90b846bfb2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.439672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01327-6a56-4d7a-a7a9-90b846bfb2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.450061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:28.465350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-dab01327-6a56-4d7a-a7a9-90b846bfb2b4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:28.466722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-dab01327-6a56-4d7a-a7a9-90b846bfb2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:28.529569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1297/2559] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:28.539394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:28.541408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:28.575185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:28.605122 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d25c3f-abae-4548-8dbc-08dd72c8c7b6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:28.609232 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:28.609881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:28.610160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:28.610428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:28.619839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:28.621324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:28.630910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1263/2560] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:29.240169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdcc0b-9734-410a-995f-d5fd89fc8f37 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:29.240785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdcc0b-9734-410a-995f-d5fd89fc8f37 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:29.240785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdcc0b-9734-410a-995f-d5fd89fc8f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:29.269364 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e8bdcc0b-9734-410a-995f-d5fd89fc8f37 None None] MFA Rules not processed for user `ddb9e04ba5c046398965b7e79eb3280b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:29.270627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e8bdcc0b-9734-410a-995f-d5fd89fc8f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:29.338988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1298/2561] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:29.349871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:29.351269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:29.408398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823a28-bd79-4ea1-bf4b-22a971983207 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:29.412250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:29.412919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:29.413213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:29.413485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:29.423186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:29.424646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f82c0-06be-4120-934b-e2e088773765 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:29.433488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1264/2562] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:12:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:12:30.796523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:30.797101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:30.797101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:30.816730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] MFA Rules not processed for user `ddb9e04ba5c046398965b7e79eb3280b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:30.817991 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:30.826413 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1299/2563] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:39.429539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=40 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:40.811094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=60 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:12:53.225666 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:53.226139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:53.226248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:53.244514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:12:53.245764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:53.313280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1265/2564] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:12:53.363878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:12:53.365425 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:12:53.443870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b17b97-4012-40b1-b2fe-c5cebc23634a None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:12:53.449222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:12:53.449886 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:12:53.450159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:12:53.450438 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:12:53.459466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:12:53.461114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d03d2-2029-4d88-bf49-2ea54f9a0328 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:12:53.469779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1300/2565] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:12:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:13:03.304819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=25 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:13:03.466447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=32 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:13:10.224209 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:10.226866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:10.305850 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc858-d2ba-4a55-acab-cb8f3396fe5c None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:10.312269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:10.312937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:10.313235 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:10.313515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:10.315506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:10.327176 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:10.328852 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383467f-883d-4f80-9f8c-0675f6cea105 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:10.339230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1266/2566] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:13:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:13:12.279305 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.279881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.279881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.281280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1301/2567] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:13:12.287153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b315-d862-43e6-a511-e27e75096d3e None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.287637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b315-d862-43e6-a511-e27e75096d3e None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.287824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b315-d862-43e6-a511-e27e75096d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.306821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8790b315-d862-43e6-a511-e27e75096d3e None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:12.308151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8790b315-d862-43e6-a511-e27e75096d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:12.377036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1267/2568] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:12.388953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:12.391155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:12.458386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c4c38-b3df-4d75-8f30-1e16ecd68efb None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:12.462673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5e14d9d362461dae34b8330718002b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:12.463344 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.463621 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.463994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.476572 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:12.478145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4b5ba-597a-4517-bc30-f52ac0db1db8 service glance] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:12.490558 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1302/2569] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:13:12.566455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.566864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.567042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] PATH_INFO: `/` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.568531 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1268/2570] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:13:12.574115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a2df-75c2-4a42-8be6-c713d18dbe74 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.574511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a2df-75c2-4a42-8be6-c713d18dbe74 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.574870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a2df-75c2-4a42-8be6-c713d18dbe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.597058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0993a2df-75c2-4a42-8be6-c713d18dbe74 None None] MFA Rules not processed for user `872c307ac6c24374b76b61b905d1f4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:12.598311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0993a2df-75c2-4a42-8be6-c713d18dbe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:12.605277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1303/2571] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:12.618322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:12.620156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:12.636213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29494bf7-c1c4-4656-9c2e-c570207c6f34 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:12.641228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:12.642182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.642394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.642786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.650924 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:12.652393 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688175c-5e93-4373-8dbc-d4a18dd39c21 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:12.661827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1269/2572] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:13:12.749298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198330-05d0-4d69-bff1-b013dd320899 None None] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.749488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198330-05d0-4d69-bff1-b013dd320899 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.749783 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198330-05d0-4d69-bff1-b013dd320899 None None] PATH_INFO: `/` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.751397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1304/2573] 10.222.0.55 () {54 vars in 921 bytes} [Tue Jun 28 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 28 12:13:12.756938 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2da3-43a2-4564-9a0d-6118c9da15ef None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.757326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2da3-43a2-4564-9a0d-6118c9da15ef None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.757591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2da3-43a2-4564-9a0d-6118c9da15ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.777474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-501b2da3-43a2-4564-9a0d-6118c9da15ef None None] MFA Rules not processed for user `77cc74fa2a6c48d0b85e3b2e70f451f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:12.778678 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-501b2da3-43a2-4564-9a0d-6118c9da15ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:12.788806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1270/2574] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Jun 28 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:12.800348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198330-05d0-4d69-bff1-b013dd320899 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:12.812943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198330-05d0-4d69-bff1-b013dd320899 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:12.817892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:12.818537 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:12.818827 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:12.819100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:12.830370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:12.831797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:12.842758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1305/2575] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:13:14.637693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a1ea-2a42-4405-aea3-a30dfbef2701 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:14.638221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a1ea-2a42-4405-aea3-a30dfbef2701 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:14.638221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a1ea-2a42-4405-aea3-a30dfbef2701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:14.657304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-72c3a1ea-2a42-4405-aea3-a30dfbef2701 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:14.658512 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-72c3a1ea-2a42-4405-aea3-a30dfbef2701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:14.666796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1271/2576] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:14.679081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:14.691587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896685-8fcc-47a0-a226-1e034809d798 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:14.698310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:14.698959 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:14.699242 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:14.699518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:14.709658 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:14.711269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c677456-baf9-44dd-9865-9d95f369e9a4 service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:14.719424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1306/2577] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:13:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:13:24.651982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=75 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:13:24.716121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=41 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:13:45.490219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:45.490767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:45.490767 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:45.510135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] MFA Rules not processed for user `9f56095d6bac4f108a73790ac9004f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:45.511333 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:45.573450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1272/2578] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:13:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:46.346903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ceb6a-d1ec-4b5e-b82b-f8043cd7150b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:46.347393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ceb6a-d1ec-4b5e-b82b-f8043cd7150b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:46.347580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ceb6a-d1ec-4b5e-b82b-f8043cd7150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:46.371682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d59ceb6a-d1ec-4b5e-b82b-f8043cd7150b None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:13:46.373060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d59ceb6a-d1ec-4b5e-b82b-f8043cd7150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:13:46.380611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1307/2579] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:13:46.394334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:13:46.415751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c775-0ee7-4403-bf3a-8612fbef5dbb None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:13:46.424829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:13:46.425550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:13:46.425836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:13:46.426112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:13:46.434888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:13:46.436569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:13:46.444893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1273/2580] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:13:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:13:56.364288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:13:56.441841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=31 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:11.123660 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:11.124254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:11.124254 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:11.144066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] MFA Rules not processed for user `ddb9e04ba5c046398965b7e79eb3280b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:11.145317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:11.152845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1308/2581] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:16.479797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:16.490146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5ebb-2b63-491d-9828-046742d6ce0c service swift] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:16.494128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:16.494769 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:16.495045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:16.495316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:16.497455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:16.566137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:16.567974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:16.578899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1274/2582] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:21.137613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:24.856298 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:24.866593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e5a53-424a-4d93-92f1-b2ddb0f2d01c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:24.870523 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:24.871205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:24.871486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:24.871760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:24.873665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:24.887563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:24.889028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:24.900185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1309/2583] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:26.574073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=30 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:26.920249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:26.929684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5f2ef-ec3d-4f53-8058-b5bab7a52cae service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:26.934415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:26.935091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:26.935389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:26.935669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:26.937837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:26.950900 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:26.952373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:26.963934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1275/2584] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:30.241143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:30.243742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:30.258035 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ba227-8ddf-4ae4-a32b-2b6ec643280c service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:30.262981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:30.263961 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.264245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.264518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] PATH_INFO: `/v3/users/b8d8d0c23f744a7b9ad311581f6ae39f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.279909 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d8d0c23f744a7b9ad311581f6ae39f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:30.281931 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:30.343554 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d8d0c23f744a7b9ad311581f6ae39f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.354450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d8d0c23f744a7b9ad311581f6ae39f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.362113 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d8d0c23f744a7b9ad311581f6ae39f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.363099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d8d0c23f744a7b9ad311581f6ae39f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.363740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e6c111fd-2162-46ae-bc69-7a827d9b69cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d8d0c23f744a7b9ad311581f6ae39f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.382094 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1310/2585] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:14:30 2022] DELETE /identity/v3/users/b8d8d0c23f744a7b9ad311581f6ae39f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:14:30.571160 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:30.582248 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a13d0-b1d1-497d-874d-baa1c2d158e8 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:30.642980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:30.643851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.644127 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.644398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] PATH_INFO: `/v3/projects/73679a9c1e1140dab3237b74ff50d3c8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.656047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73679a9c1e1140dab3237b74ff50d3c8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:30.658064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:30.716635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73679a9c1e1140dab3237b74ff50d3c8', 'request_id': 'req-64d27404-bcab-46c0-a7b2-5c3ede96adc8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.723556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73679a9c1e1140dab3237b74ff50d3c8', 'request_id': 'req-64d27404-bcab-46c0-a7b2-5c3ede96adc8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.730149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73679a9c1e1140dab3237b74ff50d3c8', 'request_id': 'req-64d27404-bcab-46c0-a7b2-5c3ede96adc8'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.744153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] The token cache is being invalidate because project 73679a9c1e1140dab3237b74ff50d3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:14:30.744456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.745116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-64d27404-bcab-46c0-a7b2-5c3ede96adc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:14:30.746721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1276/2586] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:14:30 2022] DELETE /identity/v3/projects/73679a9c1e1140dab3237b74ff50d3c8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:14:30.752062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.752373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.752649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.778531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:30.778531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:30.835244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1311/2587] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:30.843797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.844082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.844384 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.863140 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:30.864275 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:30.873606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1277/2588] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:30.877890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:30.887995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7469d52-6893-4727-9ee9-41773ab9a04b None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:30.892986 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:30.893645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.893925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.894204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.894926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:14:30.895230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:30.896858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:30.908863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1312/2589] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:14:30.913695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:30.928398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467a5df-be4c-416c-a95b-5d05bcd6dbab None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:30.937149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:30.938033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:30.938403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:30.938740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:30.939818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:30.942825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:30.976966 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1278/2590] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:14:30 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:14:30.981286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:30.991627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd419501-7d9d-42b7-be98-3273cb71fb8b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:31.026881 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:31.054003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:31.054685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.054969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.055244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.055946 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:31.057944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:31.079053 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1313/2591] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:14:30 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:14:31.083831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:31.094727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8b78-0895-40b2-ba39-4c2d0bde8a9a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:31.100020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:31.100741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.101050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.101349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.102072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:31.103716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:31.114466 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1279/2592] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:14:31.118743 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:31.128958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac2ae0-d939-4fa3-ac2c-6d1d7fe88b46 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:31.133766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:31.134429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.134715 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.134993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.135748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:31.137411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd0e7-77ec-4631-baf2-ce0dc1c53e88 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:31.147460 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1314/2593] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:14:31.151609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:31.161973 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8350-238e-4051-ae53-578adf292c73 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:31.167965 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:31.168859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.169162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.169434 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] PATH_INFO: `/v3/projects/6cf87482b69e40dda5b8104e3e1b12e8/users/f2f9826f53b44e1bbaf721d66de1356b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.184637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf87482b69e40dda5b8104e3e1b12e8, user_id=f2f9826f53b44e1bbaf721d66de1356b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:31.187359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:31.201810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1280/2594] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:14:31 2022] PUT /identity/v3/projects/6cf87482b69e40dda5b8104e3e1b12e8/users/f2f9826f53b44e1bbaf721d66de1356b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:14:31.207785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88680df6-3a79-4668-974e-c7e5822a98f2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.208078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88680df6-3a79-4668-974e-c7e5822a98f2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.208352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88680df6-3a79-4668-974e-c7e5822a98f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.228390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-88680df6-3a79-4668-974e-c7e5822a98f2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:31.229665 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-88680df6-3a79-4668-974e-c7e5822a98f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:31.301771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1315/2595] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:31.312476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:31.314403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:31.322214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:31.374889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67989b-5ad0-4572-96ff-cd0c9e810fc5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:31.379278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:31.379932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:31.380213 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:31.380514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:31.390749 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:31.392216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:31.402378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1281/2596] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:35.679114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c84a5c-af92-4459-ba50-710eb87a0a4f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:35.679691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c84a5c-af92-4459-ba50-710eb87a0a4f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:35.679691 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c84a5c-af92-4459-ba50-710eb87a0a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:35.716163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-e6c84a5c-af92-4459-ba50-710eb87a0a4f None None] MFA Rules not processed for user `f2f9826f53b44e1bbaf721d66de1356b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:35.718002 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e6c84a5c-af92-4459-ba50-710eb87a0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:35.808134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1316/2597] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:35.822993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:35.824459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:35.910688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c332bb-c222-43da-9ddd-f9c36389d3bb service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:35.916380 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:35.917096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:35.917378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:35.917653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:35.925859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:35.927469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:35.936823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1282/2598] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:37.254708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:37.255585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:37.255859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:37.275318 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] MFA Rules not processed for user `f2f9826f53b44e1bbaf721d66de1356b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:14:37.276795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:37.283852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1317/2599] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:14:37.982519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:37.994824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a5f1-53f8-4625-b527-8f2d2fef6f48 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:37.999295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:38.000154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:38.000381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:38.000684 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:38.002516 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:38.018794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:38.020791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:38.033708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1283/2600] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:14:47.269417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:48.027823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=64 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:14:57.159370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:14:57.160933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:57.173950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f5521-be1b-4d88-827a-e8973272f6ee None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:14:57.177839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:14:57.178581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:14:57.178863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:14:57.179133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:14:57.181081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:14:57.191664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:14:57.193154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:14:57.201699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1318/2601] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:14:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:15:04.976020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:04.978701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:05.041764 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9ab0-c9d1-4f50-a4bf-5b585d0a985e service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:05.047074 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:05.047736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:05.048012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:05.048280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:05.050475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:05.133919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:05.135382 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:05.155269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1284/2602] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:07.197944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:08.556929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:08.558725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:08.577359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598c3b-6dd5-4647-8a5e-329ee37968a7 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:08.588021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:08.589179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:08.590637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:08.591228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:08.593559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:08.608442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:08.610203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:08.625175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1319/2603] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:15.152048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=61 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:16.628001 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:16.637914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20841d-a548-425d-acbb-fde4c3f6e0d7 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:16.641916 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:16.642552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:16.642829 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:16.643059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:16.645135 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:16.657409 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:16.658844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:16.668935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1285/2604] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:18.618916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=6 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:19.944299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:19.945816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:19.961163 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d823a-b7cf-412a-8bf6-15b5721e53bc service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:19.966114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:19.967032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:19.967310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:19.967614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] PATH_INFO: `/v3/users/702dcd49668c4680b5bf3c1c75575ab7` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:19.968872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=702dcd49668c4680b5bf3c1c75575ab7)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:19.970882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.020800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702dcd49668c4680b5bf3c1c75575ab7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.028228 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702dcd49668c4680b5bf3c1c75575ab7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.035431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702dcd49668c4680b5bf3c1c75575ab7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.036270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702dcd49668c4680b5bf3c1c75575ab7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.036947 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2c733d11-be71-46d2-884b-9bcbe54d5bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702dcd49668c4680b5bf3c1c75575ab7'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.054410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1320/2605] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:19 2022] DELETE /identity/v3/users/702dcd49668c4680b5bf3c1c75575ab7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:20.237614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.248613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e780a-f0d5-4ab3-b528-4630391e7e98 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.309658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.310537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.310816 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.311087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] PATH_INFO: `/v3/projects/cb27799cde2e4258a2eedc5bda555c61` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.312395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb27799cde2e4258a2eedc5bda555c61)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.314335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.362948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb27799cde2e4258a2eedc5bda555c61', 'request_id': 'req-841b55c4-9eb7-452a-9794-f569c7f81935'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.369069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb27799cde2e4258a2eedc5bda555c61', 'request_id': 'req-841b55c4-9eb7-452a-9794-f569c7f81935'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.375041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb27799cde2e4258a2eedc5bda555c61', 'request_id': 'req-841b55c4-9eb7-452a-9794-f569c7f81935'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.389107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] The token cache is being invalidate because project cb27799cde2e4258a2eedc5bda555c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:20.389395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.390022 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-841b55c4-9eb7-452a-9794-f569c7f81935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:20.391615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1286/2606] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:20 2022] DELETE /identity/v3/projects/cb27799cde2e4258a2eedc5bda555c61 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:20.396681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.396999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.397278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.417174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:20.418446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:20.485932 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1321/2607] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:20.495096 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.495432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.496150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.523167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:20.524395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:20.533336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1287/2608] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:20.538160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.547963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc5229-b5c1-4745-8d48-8751698c533e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.552939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.553611 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.553893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.554168 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.554904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:20.555214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.556936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.568957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1322/2609] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:15:20.573628 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.583734 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23911e-f23a-454f-8a7f-dca6bff5e376 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.589422 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.590103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.590383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.590649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.591335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.593305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.621603 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1288/2610] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:20.626010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.637245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f5a1-a8c8-4caa-ae7b-b09d18f9b106 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.698653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.699386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.699733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.700040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.700937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.703357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.711140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:20.731202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1323/2611] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:20.732732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.744593 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dec808-afc2-42c4-8ba0-bbfb04f1a720 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.749835 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.750499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.750774 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.751039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.751763 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.753453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.766594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1289/2612] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:20.772367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.785203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bf315-188b-43ba-a243-9aeadeb5f29e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.791775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.792459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.792747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.793050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.793799 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.795421 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b7cfb-b260-4ca7-8f85-3169d90c08c8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.806530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1324/2613] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:20.811014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:20.821786 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848646-70fa-40ec-a2ea-c9a209ae3db8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:20.827301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:20.828243 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.828526 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.828853 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] PATH_INFO: `/v3/projects/034b374408c740d48e9d04bb26d707e9/users/e80262e90bd84a82a6a043c73ab466b8/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.847276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034b374408c740d48e9d04bb26d707e9, user_id=e80262e90bd84a82a6a043c73ab466b8, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:20.850788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:20.866953 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1290/2614] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:20 2022] PUT /identity/v3/projects/034b374408c740d48e9d04bb26d707e9/users/e80262e90bd84a82a6a043c73ab466b8/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:20.877591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:20.877879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:20.878214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:20.910805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] MFA Rules not processed for user `e80262e90bd84a82a6a043c73ab466b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:20.912060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:20.984151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1325/2615] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:20.989337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.003199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:21.056875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332ff4-6203-40f9-9811-5069ea4c8ea3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.062075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.062773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.063058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.063332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.064050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.066105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.093082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1291/2616] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:20 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:21.097518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.109316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7394a-bf3b-4821-af17-895e6700f26d None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.167058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.167739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.168026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.168303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.169040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.170983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.193156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1326/2617] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:21 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:21.199354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.212323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bfcf2-d156-4308-923a-07940b6dd5d2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.218201 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.218874 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.219153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.219423 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.220146 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.221795 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.232989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1292/2618] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:21.237490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.251849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f680-321a-4cbf-ab76-8207842019df admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.259017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.259925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.260430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.260848 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] PATH_INFO: `/v3/projects/93a69246a9214bc49b4e89d2bdab99a2/users/8dc39ab749a5445c802c934564611807/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.281681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a69246a9214bc49b4e89d2bdab99a2, user_id=8dc39ab749a5445c802c934564611807, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.285738 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.303963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1327/2619] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:21 2022] PUT /identity/v3/projects/93a69246a9214bc49b4e89d2bdab99a2/users/8dc39ab749a5445c802c934564611807/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:21.308200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.318876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f94b5-4f22-449e-bd29-d88e6cb77111 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.389247 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.389882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.390139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.390435 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.391161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.392772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cab05a-5ae7-43a5-9140-ca1469552f61 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.402788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1293/2620] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:21.407167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.417813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4259d-7e4f-41be-81b8-52ac3e7f58a4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.422851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.423745 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.424026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.424302 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] PATH_INFO: `/v3/projects/93a69246a9214bc49b4e89d2bdab99a2/users/8dc39ab749a5445c802c934564611807/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.426595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a69246a9214bc49b4e89d2bdab99a2, user_id=8dc39ab749a5445c802c934564611807, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.429313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.439960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1328/2621] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:21 2022] PUT /identity/v3/projects/93a69246a9214bc49b4e89d2bdab99a2/users/8dc39ab749a5445c802c934564611807/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:21.450386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.450656 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.450940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.482747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] MFA Rules not processed for user `8dc39ab749a5445c802c934564611807`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:21.484060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:21.565257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1294/2622] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:21.579893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.582070 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:21.608385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:21.647605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8fbce-c93c-4c06-8db0-78f35fd0d075 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.651281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.652017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.652275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.652511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.662133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.663587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.674940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1329/2623] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:21.719332 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:21.729768 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0a847-2053-424d-8ac7-725294b662d3 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:21.733554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:21.734200 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:21.734472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:21.734740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:21.746737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:21.805826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:21.807276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:21.816278 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1295/2624] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:24.479102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:24.480517 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:24.549755 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae3fce-b4b8-4bf5-ae90-2bf5ea21b85d service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:24.555439 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:24.556372 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:24.556647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:24.556942 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] PATH_INFO: `/v3/users/ddb9e04ba5c046398965b7e79eb3280b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:24.558273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb9e04ba5c046398965b7e79eb3280b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:24.560223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:24.612080 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb9e04ba5c046398965b7e79eb3280b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.621403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb9e04ba5c046398965b7e79eb3280b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.628776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb9e04ba5c046398965b7e79eb3280b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.629543 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb9e04ba5c046398965b7e79eb3280b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.630232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1e94b5ee-b28b-4ae8-a7ba-ca822adc96cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb9e04ba5c046398965b7e79eb3280b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.648102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1330/2625] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:24 2022] DELETE /identity/v3/users/ddb9e04ba5c046398965b7e79eb3280b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:24.831708 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:24.843748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465199-0c27-4459-b1b7-bd6029f9fe8e service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:24.915340 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:24.916356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:24.916663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:24.917048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] PATH_INFO: `/v3/projects/2ac1aab84e994f7892964ff85819002d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:24.918240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac1aab84e994f7892964ff85819002d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:24.920156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:24.968761 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac1aab84e994f7892964ff85819002d', 'request_id': 'req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.976831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac1aab84e994f7892964ff85819002d', 'request_id': 'req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.982890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac1aab84e994f7892964ff85819002d', 'request_id': 'req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.997373 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] The token cache is being invalidate because project 2ac1aab84e994f7892964ff85819002d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:24.997633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.998229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a6053cb9-d27e-40c4-9f17-e906cedd1e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:24.999819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1296/2626] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:24 2022] DELETE /identity/v3/projects/2ac1aab84e994f7892964ff85819002d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:25.007451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.007739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.007978 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.025635 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:25.026847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.068740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:25.095815 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1331/2627] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:25.104426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.104810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.105244 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.124298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:25.125540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.134371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1297/2628] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:25.138860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.150090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8d479-e1d3-4cdc-b79a-9b918c791587 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.154897 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.155645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.155996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.156338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.157179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:25.157556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.159435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.173429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1332/2629] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:25.178097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.189720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.191925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.193739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92656-f7c3-4875-9d6a-4525c58333be None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.198417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.199084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.199359 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.199627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.200300 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.202658 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.206098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ef54d-668a-46a3-8cc2-8581979f2b9d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.211335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.212223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.212500 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.212775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] PATH_INFO: `/v3/users/e80262e90bd84a82a6a043c73ab466b8` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.213993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80262e90bd84a82a6a043c73ab466b8)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.215900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.231066 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1298/2630] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:25.235107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.246369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:25.268884 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80262e90bd84a82a6a043c73ab466b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.276785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80262e90bd84a82a6a043c73ab466b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.285531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80262e90bd84a82a6a043c73ab466b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.286332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80262e90bd84a82a6a043c73ab466b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.286962 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3c817cf2-1ed7-4de1-a3cd-6716f16ad4bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80262e90bd84a82a6a043c73ab466b8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.304526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1333/2631] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:25 2022] DELETE /identity/v3/users/e80262e90bd84a82a6a043c73ab466b8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:25.305397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb65e-ad9a-41c6-b9b6-753ec9364fc9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.309015 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.309309 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.309588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.326793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:25.328003 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.364051 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.364720 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.365100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.365485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.366049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.368033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.388958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1334/2632] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:25.389277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1299/2633] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:25.393036 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.394444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.398499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.400221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.406905 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33055555-2b35-4396-ba7d-6eb5c029c9e3 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.412063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.412725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.413039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.413313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.414041 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.415659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.425675 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1335/2634] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:25.429589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.439020 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724c72a-4f30-4afc-b687-a6270d0ee652 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.443807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.444459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.444739 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.445052 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.445766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.447367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.461822 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1336/2635] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:25.462006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76825e-6a3f-4783-a7fe-46a7a1ef4506 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.462813 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.463698 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.464344 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.464625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.464955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.472241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d99246-eb07-4d1f-8a02-a80ba3f607c5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.474471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.475926 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1169202-0e81-4ca1-8528-5b6cb352f48c service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.477409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.478304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.478606 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.478880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] PATH_INFO: `/v3/projects/56f6e9f5c280479d9050a6bfdde0d68b/users/1c0c3b5771de4561835b6bb4a74c86ed/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.485885 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1300/2636] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:15:25.494279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f6e9f5c280479d9050a6bfdde0d68b, user_id=1c0c3b5771de4561835b6bb4a74c86ed, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.497374 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.500930 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:25.514263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1337/2637] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:25 2022] PUT /identity/v3/projects/56f6e9f5c280479d9050a6bfdde0d68b/users/1c0c3b5771de4561835b6bb4a74c86ed/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:25.519540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.519799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.520172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.539112 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:25.540365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.590255 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:25.607061 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1301/2638] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:25.617393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.619678 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.694869 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d6903-f5d3-4a22-9e80-922d5e020ffe admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.699131 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.699800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.700078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.700352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.710206 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.711940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.722179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1338/2639] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:25.802222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.803930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:25.816143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e47a1d-e4b8-4d3f-a518-78bec8847bce None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:25.830024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:25.831291 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:25.831654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:25.832951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] PATH_INFO: `/v3/projects/034b374408c740d48e9d04bb26d707e9` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:25.833561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=034b374408c740d48e9d04bb26d707e9)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:25.837426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:25.886773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034b374408c740d48e9d04bb26d707e9', 'request_id': 'req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.893388 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034b374408c740d48e9d04bb26d707e9', 'request_id': 'req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.899781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034b374408c740d48e9d04bb26d707e9', 'request_id': 'req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.914079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] The token cache is being invalidate because project 034b374408c740d48e9d04bb26d707e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:25.914363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.914998 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:25.916857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1302/2640] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:25 2022] DELETE /identity/v3/projects/034b374408c740d48e9d04bb26d707e9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:25.920951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:25.932652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176f4af-3cf1-4415-bf2b-32dc3e628485 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:26.002680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:26.003575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.003856 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.004363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] PATH_INFO: `/v3/users/8dc39ab749a5445c802c934564611807` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.005561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc39ab749a5445c802c934564611807)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:26.007482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:26.058160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc39ab749a5445c802c934564611807'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.067221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc39ab749a5445c802c934564611807'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.075921 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc39ab749a5445c802c934564611807'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.076592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc39ab749a5445c802c934564611807'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.077331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-942a9118-feab-4eee-8840-43a518f36026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc39ab749a5445c802c934564611807'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.102762 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1339/2641] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:25 2022] DELETE /identity/v3/users/8dc39ab749a5445c802c934564611807 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:26.483027 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:26.484686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:26.507889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272c66f-5b1b-4c1f-8626-3693e8ab8cd7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:26.583917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:26.584831 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.585307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.585619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] PATH_INFO: `/v3/projects/93a69246a9214bc49b4e89d2bdab99a2` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.586931 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a69246a9214bc49b4e89d2bdab99a2)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:26.588941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:26.648073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a69246a9214bc49b4e89d2bdab99a2', 'request_id': 'req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.655864 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a69246a9214bc49b4e89d2bdab99a2', 'request_id': 'req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.663374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a69246a9214bc49b4e89d2bdab99a2', 'request_id': 'req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.680084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] The token cache is being invalidate because project 93a69246a9214bc49b4e89d2bdab99a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:26.680699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.681567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3dd504e2-aa9e-4658-b0f1-11c1621d4d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:26.683606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1303/2642] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:26 2022] DELETE /identity/v3/projects/93a69246a9214bc49b4e89d2bdab99a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:26.689146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.689816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.689926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.710615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:26.712019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:26.727101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:26.785531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1340/2643] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:26.794257 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.794547 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.794824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.802163 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:26.816087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:26.818560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:26.828088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1304/2644] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:26.832937 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:26.843893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13789c-2cb6-4c90-8832-662cedd0ca57 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:26.848798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:26.849916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.849916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.850142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.850845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:26.851154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:26.852800 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:26.866490 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1341/2645] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:26.871011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:26.882107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aad482-ad43-4efa-92d2-25ed69abf85a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:26.888088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:26.889005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:26.889385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:26.889701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:26.890599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:26.892568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:26.921758 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1305/2646] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:26 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:26.926097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:26.937680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeb557-8da0-46a0-b638-45d874358be5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.009411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.010109 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.010390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.010708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.011390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.013355 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.040808 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1342/2647] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:26 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:27.042378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.055733 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9eeda-ca25-44b9-ab2a-80606de34319 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.061532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.062759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.063755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.064765 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.066032 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.068216 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.086576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1306/2648] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:27.094752 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.105552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514f9a5-91f5-488b-b19e-75d39457c867 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.110747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.111539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.111920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.112639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.113877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.115644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c8f03-8bb7-41fc-a616-a19068effa1e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.126874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1343/2649] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:27.131238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.142955 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24539fdf-1f87-4d80-8614-8dd1542fe190 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.147980 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.148887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.149197 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.149809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] PATH_INFO: `/v3/projects/be8941d959574d7eb5792509fd45f8d1/users/acd455873d82496f9f50c12bf2af2849/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.166208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8941d959574d7eb5792509fd45f8d1, user_id=acd455873d82496f9f50c12bf2af2849, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.168502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.185073 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1307/2650] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:27 2022] PUT /identity/v3/projects/be8941d959574d7eb5792509fd45f8d1/users/acd455873d82496f9f50c12bf2af2849/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:27.197270 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.198402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.198688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.243403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] MFA Rules not processed for user `acd455873d82496f9f50c12bf2af2849`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:27.245494 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:27.316102 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1344/2651] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:27.321211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.417798 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a56fa-c8a0-47d8-a32d-72002da831b8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.428616 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.429355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.429642 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.429915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.430762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.433455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.463895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1308/2652] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:27 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:27.467805 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.480240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472295b-3ce1-427c-a957-b4e63ba38d36 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.493728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:27.552260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.553263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.553547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.553825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.554542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.556482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.578757 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1345/2653] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:27 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:27.583191 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.594287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615b42e-b52b-41c5-9498-49afbaca8833 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.599494 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.600334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.600690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.601063 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.602025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.603778 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.615258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1309/2654] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:27.619385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.630085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af0e02-7ad6-429a-8845-f55692bc6ffb admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.634904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.635809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.636150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.636363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] PATH_INFO: `/v3/projects/b02098af01a9410ca4309cc71d52ac80/users/7cf7d091d38a439180221eda66016087/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.655851 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02098af01a9410ca4309cc71d52ac80, user_id=7cf7d091d38a439180221eda66016087, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.660948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.686680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1346/2655] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:27 2022] PUT /identity/v3/projects/b02098af01a9410ca4309cc71d52ac80/users/7cf7d091d38a439180221eda66016087/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:27.690677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.701811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2edbce-e3d2-4bdc-9381-88756ccc14fb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.770911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.771911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.772897 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.773367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.774253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.776049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99138fc-1b42-4689-b207-e5d907ea8b0e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.787968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1310/2656] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:27.792975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.803875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99edce6-89bc-434c-9e82-990a5ef2d71e admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:27.809346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:27.810259 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.810540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.810812 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] PATH_INFO: `/v3/projects/b02098af01a9410ca4309cc71d52ac80/users/7cf7d091d38a439180221eda66016087/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.813683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02098af01a9410ca4309cc71d52ac80, user_id=7cf7d091d38a439180221eda66016087, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:27.816388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:27.829218 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1347/2657] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:27 2022] PUT /identity/v3/projects/b02098af01a9410ca4309cc71d52ac80/users/7cf7d091d38a439180221eda66016087/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:27.841071 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:27.841368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:27.841643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:27.867058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:27.879773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] MFA Rules not processed for user `7cf7d091d38a439180221eda66016087`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:27.882015 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:27.964227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1311/2658] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:27.977151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:27.978916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:28.058444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7df7-149b-44ef-b9a0-4c59a0d87d5f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:28.063330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:28.064262 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:28.064842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:28.065307 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:28.078358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:28.080451 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:28.094487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1348/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:28.506946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:28.517594 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d319-2e8d-4b6c-9b8d-29f087ede0ce None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:28.522879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf7d091d38a439180221eda66016087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b02098af01a9410ca4309cc71d52ac80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:28.523609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:28.523901 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:28.524183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:28.524941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:28.525252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:28.528564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:28.548520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1312/2660] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:28.553885 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:28.563358 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbccdd2-2ea4-4996-8b16-64e53293aed2 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:28.569407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf7d091d38a439180221eda66016087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b02098af01a9410ca4309cc71d52ac80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:28.570090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:28.570380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:28.570662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:28.571501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:28.573510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:28.627446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1349/2661] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:28 2022] POST /identity/v3/projects => generated 352 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:28.631619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:28.646867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e8e8-97e9-420c-a72f-997710a9d863 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:28.734557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf7d091d38a439180221eda66016087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b02098af01a9410ca4309cc71d52ac80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:28.734806 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:28.735298 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:28.735664 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:28.736750 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:28.737289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:28.739478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:28.757002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1313/2662] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:28.806971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:28.818417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721797c3-6fea-45d2-8d05-878d308c53fa tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:28.823200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf7d091d38a439180221eda66016087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b02098af01a9410ca4309cc71d52ac80', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:28.824375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:28.824742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:28.825114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] PATH_INFO: `/v3/projects/4c970bc232384a15af879be4d950cba4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:28.826768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4c970bc232384a15af879be4d950cba4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:28.828721 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:28.878601 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:28.885970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c970bc232384a15af879be4d950cba4', 'request_id': 'req-1babbb39-93f2-475e-a091-02f5c3025b44'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:28.894650 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c970bc232384a15af879be4d950cba4', 'request_id': 'req-1babbb39-93f2-475e-a091-02f5c3025b44'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:28.902547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c970bc232384a15af879be4d950cba4', 'request_id': 'req-1babbb39-93f2-475e-a091-02f5c3025b44'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:28.916416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] The token cache is being invalidate because project 4c970bc232384a15af879be4d950cba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:28.916649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:28.917316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-1babbb39-93f2-475e-a091-02f5c3025b44 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:28.919863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1350/2663] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:28 2022] DELETE /identity/v3/projects/4c970bc232384a15af879be4d950cba4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:29.154410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:29.156305 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:29.235680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ba07f-4449-4b81-99b1-c85087fbb9d7 tempest-VolumeQuotaClassesTest-1624297346 tempest-VolumeQuotaClassesTest-1624297346-project-admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:29.259667 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:29.310406 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:29.312128 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:29.312507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:29.313220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] PATH_INFO: `/v3/users/acd455873d82496f9f50c12bf2af2849` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:29.315038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd455873d82496f9f50c12bf2af2849)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:29.317367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:29.384647 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd455873d82496f9f50c12bf2af2849'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:29.393692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd455873d82496f9f50c12bf2af2849'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:29.405369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd455873d82496f9f50c12bf2af2849'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:29.406256 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd455873d82496f9f50c12bf2af2849'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:29.407132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd455873d82496f9f50c12bf2af2849'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:29.429161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1314/2664] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:29 2022] DELETE /identity/v3/users/acd455873d82496f9f50c12bf2af2849 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:29.434456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:29.434747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:29.435048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:29.464467 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:29.465877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:29.533971 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1351/2665] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:29.543886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:29.545532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:29.622266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb67b2-c356-4aad-8024-95eb47b74091 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:29.630054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:29.631000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:29.631270 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:29.631732 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:29.646079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:29.648322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:29.671568 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1315/2666] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:30.197960 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:30.199662 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:30.214414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8cf8e-9ac8-4650-aa91-8b5518a0cbd4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:30.221652 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:30.222563 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.223055 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.224526 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] PATH_INFO: `/v3/projects/be8941d959574d7eb5792509fd45f8d1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.226578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8941d959574d7eb5792509fd45f8d1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:30.229623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:30.299009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8941d959574d7eb5792509fd45f8d1', 'request_id': 'req-e9704908-eb9c-4365-b986-d98a03961737'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.307335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8941d959574d7eb5792509fd45f8d1', 'request_id': 'req-e9704908-eb9c-4365-b986-d98a03961737'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.314006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8941d959574d7eb5792509fd45f8d1', 'request_id': 'req-e9704908-eb9c-4365-b986-d98a03961737'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.334680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] The token cache is being invalidate because project be8941d959574d7eb5792509fd45f8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:30.334994 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.335641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e9704908-eb9c-4365-b986-d98a03961737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.338050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1352/2667] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:30 2022] DELETE /identity/v3/projects/be8941d959574d7eb5792509fd45f8d1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:30.342504 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:30.355912 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603c351-f357-4183-bcd8-7576c806cdc2 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:30.413374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:30.439212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:30.440445 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.441454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.442687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] PATH_INFO: `/v3/users/7cf7d091d38a439180221eda66016087` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.445227 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf7d091d38a439180221eda66016087)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:30.447363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:30.517354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf7d091d38a439180221eda66016087'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.527877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf7d091d38a439180221eda66016087'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.543133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf7d091d38a439180221eda66016087'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.543879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf7d091d38a439180221eda66016087'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.544578 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf7d091d38a439180221eda66016087'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:30.575969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1316/2668] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:30 2022] DELETE /identity/v3/users/7cf7d091d38a439180221eda66016087 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:30.602546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.603075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.603453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.643988 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] MFA Rules not processed for user `1c0c3b5771de4561835b6bb4a74c86ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:30.645681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:30.703202 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:30.738386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1353/2669] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:30.743574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:30.746130 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:30.843740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32237101-1268-42a8-aff4-82613c4b2a2a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:30.850432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:30.851271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.851648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.852013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.852818 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:30.855101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:30.881817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1317/2670] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:30 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:30.886339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:30.898682 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b093e-e20a-4c2a-bedb-54bfa5350f09 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:30.909530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:30.911369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:30.967006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:30.970641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:30.971348 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.971719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.971904 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.972771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:30.974945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:30.984648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c4e9-7192-4152-b1b2-bde74c5ac498 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:30.991404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:30.992281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:30.992560 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:30.992832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] PATH_INFO: `/v3/projects/b02098af01a9410ca4309cc71d52ac80` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:30.994068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02098af01a9410ca4309cc71d52ac80)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:30.998929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1354/2671] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:30 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:30.999092 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.000022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.010882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6555d-0edb-45fc-b3c6-4b4758d17db7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.016189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.017147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.017579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.018034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.018934 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.021195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.036836 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1355/2672] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:31.037880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.048999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3798a51-73f0-464b-8449-fbf413cf351d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.054020 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02098af01a9410ca4309cc71d52ac80', 'request_id': 'req-0c1a05bc-423d-42a5-8529-22b7d8346d90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:31.060696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02098af01a9410ca4309cc71d52ac80', 'request_id': 'req-0c1a05bc-423d-42a5-8529-22b7d8346d90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:31.068116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02098af01a9410ca4309cc71d52ac80', 'request_id': 'req-0c1a05bc-423d-42a5-8529-22b7d8346d90'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:31.089454 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] The token cache is being invalidate because project b02098af01a9410ca4309cc71d52ac80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:31.089802 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:31.090939 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-0c1a05bc-423d-42a5-8529-22b7d8346d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:31.092979 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1318/2673] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:30 2022] DELETE /identity/v3/projects/b02098af01a9410ca4309cc71d52ac80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:31.097625 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a56f81-d8d4-4d2d-8d35-aa56bdbad8ea None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.097946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a56f81-d8d4-4d2d-8d35-aa56bdbad8ea None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.098221 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a56f81-d8d4-4d2d-8d35-aa56bdbad8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.118229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-31a56f81-d8d4-4d2d-8d35-aa56bdbad8ea None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:31.119459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-31a56f81-d8d4-4d2d-8d35-aa56bdbad8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:31.124820 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.125795 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.126075 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.126346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] PATH_INFO: `/v3/projects/bfb4f296fbb940d3a137a41a0d340f59/users/a3a5af42b5914ab9a0b96b2238f859ea/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.141753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb4f296fbb940d3a137a41a0d340f59, user_id=a3a5af42b5914ab9a0b96b2238f859ea, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.144411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.158329 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1356/2674] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:31 2022] PUT /identity/v3/projects/bfb4f296fbb940d3a137a41a0d340f59/users/a3a5af42b5914ab9a0b96b2238f859ea/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:31.162515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.164385 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:31.182922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1319/2675] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:31.190631 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.191031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.191546 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.211240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:31.212471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:31.220173 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:31.241134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c11df-4f59-4c1a-ab70-3640df681933 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.246263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.246955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.247380 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.247664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.248452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.250132 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.262153 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1357/2676] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:31.266450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.278749 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0b2b2-d65b-486d-88ae-23c6922164dc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.282513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1320/2677] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:31.284159 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.285186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.286222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.286222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] PATH_INFO: `/v3/projects/bfb4f296fbb940d3a137a41a0d340f59/users/a3a5af42b5914ab9a0b96b2238f859ea/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.286586 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.288196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb4f296fbb940d3a137a41a0d340f59, user_id=a3a5af42b5914ab9a0b96b2238f859ea, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.291244 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.295736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ac220-1a07-4c6b-9872-d1b805a17df6 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.300530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.301956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.302241 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.302515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.303042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1358/2678] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:31 2022] PUT /identity/v3/projects/bfb4f296fbb940d3a137a41a0d340f59/users/a3a5af42b5914ab9a0b96b2238f859ea/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:31.303277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:31.303623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.305486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.321834 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1321/2679] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 28 12:15:31.323098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.396278 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73067b-47f4-40f5-897c-fe2878a52393 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.402970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.403649 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.403926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.404197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.404895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.406914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.450693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1359/2680] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:31.454641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.468308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b167-cce5-4b82-a8c3-202ecd352452 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.473714 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:31.540567 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.541374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.541711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.542084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.542844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.545002 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.570339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1322/2681] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:31.574628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.595423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a695-0bfa-4826-b1e3-3406f61c5b21 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.601600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.602370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.602741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.603201 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.604008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.605774 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.621592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1360/2682] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:31.622752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.643475 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4124-74fc-4f72-9351-6b0aa062024d admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.660420 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.661004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.663469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.663469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.663469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.663832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694aeff-2093-4a75-9651-4b00b3c15f04 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.679606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1323/2683] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:31.683542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:31.696419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8cb0d-e386-492f-ab51-b27b4cbbfe60 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:31.709121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:31.710060 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.710338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.710607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] PATH_INFO: `/v3/projects/fb72cbef7624479c8e7f4f4fe55e5fed/users/6b70921656794f90bc3e01c5c8286af9/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.751331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb72cbef7624479c8e7f4f4fe55e5fed, user_id=6b70921656794f90bc3e01c5c8286af9, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:31.754286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:31.793938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1361/2684] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:31 2022] PUT /identity/v3/projects/fb72cbef7624479c8e7f4f4fe55e5fed/users/6b70921656794f90bc3e01c5c8286af9/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:31.805621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:31.806031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:31.806426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:31.846363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] MFA Rules not processed for user `6b70921656794f90bc3e01c5c8286af9`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:31.847742 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:31.956250 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1324/2685] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:31.960550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.060540 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60781e5f-e276-4aa2-8bc1-44f37d1ce576 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.068750 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.069686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.069981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.070444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.071511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.073907 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.118679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.124804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1362/2686] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:31 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:32.134715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fddec-d449-49ed-9a46-22c7edab3e05 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.211398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.212673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.213456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.213945 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.215571 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.219044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.252641 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1325/2687] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:32 2022] POST /identity/v3/users => generated 307 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:32.253628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.259313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:32.266172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8db2-53d2-46a6-ae90-8e0f2846c5a4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.280402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.281301 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.281588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.281888 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.282977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.285135 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.297527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1363/2688] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:32.302161 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.310550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:32.320715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5811fa-1697-428f-907b-4f01b85e7691 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.327561 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.328937 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.329797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.330640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] PATH_INFO: `/v3/projects/03db92d19e50492d9d2d371ff2456909/users/e20a1012d1c0410f83cbbcb9a1e4082e/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.356402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03db92d19e50492d9d2d371ff2456909, user_id=e20a1012d1c0410f83cbbcb9a1e4082e, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.359468 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.380134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1326/2689] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:32 2022] PUT /identity/v3/projects/03db92d19e50492d9d2d371ff2456909/users/e20a1012d1c0410f83cbbcb9a1e4082e/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:32.381219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.398483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58894298-7b03-4fde-9af1-8d3c0cc93700 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.504144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.504949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.505303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.505647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.506612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.508502 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c4ae-93b8-4d28-9eec-1d5b95639153 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.540180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1364/2690] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:32.544721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.560600 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c5cd-bbc5-4ecb-943d-ff9afaf14cd5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.585119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.586274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.587212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.587659 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] PATH_INFO: `/v3/projects/03db92d19e50492d9d2d371ff2456909/users/e20a1012d1c0410f83cbbcb9a1e4082e/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.591524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03db92d19e50492d9d2d371ff2456909, user_id=e20a1012d1c0410f83cbbcb9a1e4082e, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.594623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.609654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1327/2691] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:32 2022] PUT /identity/v3/projects/03db92d19e50492d9d2d371ff2456909/users/e20a1012d1c0410f83cbbcb9a1e4082e/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:32.621703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.622149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.622684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.670492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] MFA Rules not processed for user `e20a1012d1c0410f83cbbcb9a1e4082e`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:32.672581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:32.810922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1365/2692] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:32.819919 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:32.822922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:32.935582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cb8c-ec7a-4e40-80e8-6325effb95a8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:32.945737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:32.946425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:32.946704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:32.946976 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:32.966086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:32.968318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c5d71-75d3-4de1-99a7-b1d3d71760f2 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:32.981171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1328/2693] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:33.110759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:33.123546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f2f4a-3e18-48cd-9239-764b2df2c48f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:33.130249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:33.131300 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:33.131674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:33.132039 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:33.275283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:33.276824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:33.288433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1366/2694] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:15:36.230782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9021d-0987-48d8-9116-c6938d95d393 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:36.231336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9021d-0987-48d8-9116-c6938d95d393 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:36.231336 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9021d-0987-48d8-9116-c6938d95d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:36.265623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c3d9021d-0987-48d8-9116-c6938d95d393 None None] MFA Rules not processed for user `a3a5af42b5914ab9a0b96b2238f859ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:36.267203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c3d9021d-0987-48d8-9116-c6938d95d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:36.355484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1329/2695] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:36.370073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:36.372313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:36.422703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:36.540928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f92f9d-f125-410a-9d1c-0414d221a8f2 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:36.551260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:36.552166 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:36.552566 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:36.552880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:36.555637 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:36.624488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:36.626313 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:36.636468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1367/2696] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:37.745800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac0f9-5e51-479b-a91b-83b494df7c10 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:37.747322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac0f9-5e51-479b-a91b-83b494df7c10 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:37.747753 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ac0f9-5e51-479b-a91b-83b494df7c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:37.797455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b14ac0f9-5e51-479b-a91b-83b494df7c10 None None] MFA Rules not processed for user `1c0c3b5771de4561835b6bb4a74c86ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:37.799893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:37.800591 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b14ac0f9-5e51-479b-a91b-83b494df7c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:37.813681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1330/2697] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:37.880426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728ada4-50ea-4262-a75d-4b22bb4a9a36 service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:37.885402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:37.886357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:37.886643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:37.886922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] PATH_INFO: `/v3/users/6b70921656794f90bc3e01c5c8286af9` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:37.888169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b70921656794f90bc3e01c5c8286af9)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:37.890390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:37.963406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b70921656794f90bc3e01c5c8286af9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:37.969534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:37.983125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b70921656794f90bc3e01c5c8286af9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.001115 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b70921656794f90bc3e01c5c8286af9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.005357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b70921656794f90bc3e01c5c8286af9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.007387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b70921656794f90bc3e01c5c8286af9'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.032976 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1368/2698] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:37 2022] DELETE /identity/v3/users/6b70921656794f90bc3e01c5c8286af9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:38.038084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:38.038398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:38.038677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:38.057841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:38.059169 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:38.108374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:38.131793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1331/2699] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:38.141894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:38.143437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:38.215072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728a4a7-eb30-4fe0-bd5e-8eda11dc146c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:38.219857 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:38.220632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:38.220797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:38.221099 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:38.238619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:38.240308 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:38.253672 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1369/2700] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:38.555779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:38.557696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:38.590124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a91b9-248f-4292-805f-7ead5d4cc442 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:38.595450 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:38.596346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:38.596629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:38.596922 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] PATH_INFO: `/v3/projects/fb72cbef7624479c8e7f4f4fe55e5fed` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:38.598320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb72cbef7624479c8e7f4f4fe55e5fed)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:38.600338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:38.662225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb72cbef7624479c8e7f4f4fe55e5fed', 'request_id': 'req-e3c26ad3-1867-42d9-8efa-74d250265a56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.669428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb72cbef7624479c8e7f4f4fe55e5fed', 'request_id': 'req-e3c26ad3-1867-42d9-8efa-74d250265a56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.676851 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb72cbef7624479c8e7f4f4fe55e5fed', 'request_id': 'req-e3c26ad3-1867-42d9-8efa-74d250265a56'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.692031 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] The token cache is being invalidate because project fb72cbef7624479c8e7f4f4fe55e5fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:38.692484 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.693817 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.695956 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1332/2701] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:38 2022] DELETE /identity/v3/projects/fb72cbef7624479c8e7f4f4fe55e5fed => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:38.699985 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:38.710583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e110e00-056e-436b-97af-0855629fc472 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:38.772694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:38.773642 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:38.774260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:38.774260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] PATH_INFO: `/v3/users/e20a1012d1c0410f83cbbcb9a1e4082e` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:38.775450 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20a1012d1c0410f83cbbcb9a1e4082e)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:38.777478 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:38.835241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20a1012d1c0410f83cbbcb9a1e4082e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.846829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20a1012d1c0410f83cbbcb9a1e4082e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.854760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20a1012d1c0410f83cbbcb9a1e4082e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.855456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20a1012d1c0410f83cbbcb9a1e4082e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.856072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-de2a65d7-8c57-4064-8028-0a745084985f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20a1012d1c0410f83cbbcb9a1e4082e'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:38.863617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:38.877646 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1370/2702] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:38 2022] DELETE /identity/v3/users/e20a1012d1c0410f83cbbcb9a1e4082e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:39.202403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.205437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:39.231777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c26ad3-1867-42d9-8efa-74d250265a56 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.320570 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.321929 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.322366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.322808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] PATH_INFO: `/v3/projects/03db92d19e50492d9d2d371ff2456909` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.326218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03db92d19e50492d9d2d371ff2456909)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.328374 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.388991 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03db92d19e50492d9d2d371ff2456909', 'request_id': 'req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:39.395583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03db92d19e50492d9d2d371ff2456909', 'request_id': 'req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:39.401536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03db92d19e50492d9d2d371ff2456909', 'request_id': 'req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:39.426808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] The token cache is being invalidate because project 03db92d19e50492d9d2d371ff2456909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:39.427109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:39.427836 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d4c38eda-5d84-4057-8a85-9c5bc1f62a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:39.433309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1333/2703] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:39 2022] DELETE /identity/v3/projects/03db92d19e50492d9d2d371ff2456909 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:39.435061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.435522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.435797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.461100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:39.462679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:39.550747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1371/2704] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:39.553428 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.554286 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.554797 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.568062 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:39.598152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:39.600989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:39.617654 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.617884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1334/2705] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:15:39.633165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39040c1-a13d-4240-8384-3dedde568e2f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.639610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df87867-1888-4109-9487-769134e6554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.640367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df87867-1888-4109-9487-769134e6554d admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.640719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df87867-1888-4109-9487-769134e6554d admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.641134 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df87867-1888-4109-9487-769134e6554d admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.641973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df87867-1888-4109-9487-769134e6554d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:39.642337 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df87867-1888-4109-9487-769134e6554d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.645160 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df87867-1888-4109-9487-769134e6554d admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.685783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.692522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1372/2706] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:39.701338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40334337-d5d1-4710-ae72-d22eae21f17a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.710075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.710075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.710075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.710075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.710651 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.712357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.755293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1335/2707] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:39 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:39.759390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df87867-1888-4109-9487-769134e6554d admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.790724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df87867-1888-4109-9487-769134e6554d admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.871903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.872617 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.873154 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.873434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.874182 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.876177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.903057 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1373/2708] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:39 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:39.907339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.919276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d653b5-1756-4628-89b0-c4b97b8fcc65 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.925539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.926507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.926791 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.927070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.928000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.930054 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.944621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1336/2709] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:39.948482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:39.960293 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24bee9-8bb1-4f52-951b-afe875807356 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:39.965384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:39.966406 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:39.966816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:39.967404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:39.968443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:39.970951 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7a86b-9a49-4f51-a17b-c797f3ef2436 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:39.984090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1374/2710] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:39.988639 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.003172 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f80f-682f-4cf1-b386-e8ce4e37cf18 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.009252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.010162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.010440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.010709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] PATH_INFO: `/v3/projects/186f5e50c9414335bbdae03ceff64ea8/users/551994f03add4f59b913b7e5c31563eb/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.029673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=186f5e50c9414335bbdae03ceff64ea8, user_id=551994f03add4f59b913b7e5c31563eb, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.030401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.047337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1337/2711] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:39 2022] PUT /identity/v3/projects/186f5e50c9414335bbdae03ceff64ea8/users/551994f03add4f59b913b7e5c31563eb/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:40.058923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.059280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.059624 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.097220 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] MFA Rules not processed for user `551994f03add4f59b913b7e5c31563eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:40.099013 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:40.147574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:40.172393 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1375/2712] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:40.177803 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.245760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd348ed-2317-42c7-9c91-d4c2964e3efd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.250482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.251162 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.251444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.251718 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.252412 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.255914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.284660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1338/2713] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:40 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:40.289088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.300603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4314ab-2337-49e8-859b-b1a46b34558e None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.367459 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.368145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.368893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.368893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.369534 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.371487 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.397504 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1376/2714] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:40 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:40.401715 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.415372 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138864-9d2b-46be-877c-9e3a579c55da admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.420477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.421196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.421654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.421832 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.422520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.424059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.443923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1339/2715] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:40.448265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.459341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a0b0b-9af0-46e4-a126-f8b3393a6bde admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.464151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.465118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.465404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.465699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] PATH_INFO: `/v3/projects/04ddd6953ee0434a805b38b85b16693b/users/6c404a0e6dfc4d0e93ec67b5cb27f71a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.482551 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ddd6953ee0434a805b38b85b16693b, user_id=6c404a0e6dfc4d0e93ec67b5cb27f71a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.484995 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.500260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1377/2716] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:40 2022] PUT /identity/v3/projects/04ddd6953ee0434a805b38b85b16693b/users/6c404a0e6dfc4d0e93ec67b5cb27f71a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:40.504522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.515029 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf80515-1e86-4bb1-9c7d-4734ea276a99 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.573739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.574440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.574724 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.575000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.575723 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.577407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fdffe-04ab-47b5-a513-b55dcee7e510 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.589821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1340/2717] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:40.594187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.605127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1485c-9163-4b6d-9eb3-657da2a59156 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.610071 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.610944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.611258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.611530 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] PATH_INFO: `/v3/projects/04ddd6953ee0434a805b38b85b16693b/users/6c404a0e6dfc4d0e93ec67b5cb27f71a/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.613734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ddd6953ee0434a805b38b85b16693b, user_id=6c404a0e6dfc4d0e93ec67b5cb27f71a, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.616391 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.629359 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1378/2718] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:40 2022] PUT /identity/v3/projects/04ddd6953ee0434a805b38b85b16693b/users/6c404a0e6dfc4d0e93ec67b5cb27f71a/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:40.641202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.641498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.641777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.661972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:40.673219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] MFA Rules not processed for user `6c404a0e6dfc4d0e93ec67b5cb27f71a`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:40.674481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:40.747629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1341/2719] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:40.759693 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.761679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:40.822555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3773bd-4c11-44e8-b3ea-1b6e5b2ff127 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.826521 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.827252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.827613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.827973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:40.838659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:40.840178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:40.850501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1379/2720] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:40.931335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:40.941433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc2547-c0d3-4393-8078-1a7de5c8c930 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:40.946687 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88965a-8154-4038-8178-558310922869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:40.947348 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88965a-8154-4038-8178-558310922869 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:40.947630 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88965a-8154-4038-8178-558310922869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:40.947907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88965a-8154-4038-8178-558310922869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:41.006529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88965a-8154-4038-8178-558310922869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:41.007982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88965a-8154-4038-8178-558310922869 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:41.017143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1342/2721] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:46.860619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:46.907258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:46.929006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26826c5-f1d8-44ea-9c32-07c3f31c0b3e service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:46.935263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:46.936184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:46.936462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:46.936734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] PATH_INFO: `/v3/users/551994f03add4f59b913b7e5c31563eb` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:46.938107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=551994f03add4f59b913b7e5c31563eb)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:46.940078 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:47.004255 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551994f03add4f59b913b7e5c31563eb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.012817 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551994f03add4f59b913b7e5c31563eb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.022199 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551994f03add4f59b913b7e5c31563eb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.023146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551994f03add4f59b913b7e5c31563eb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.024034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551994f03add4f59b913b7e5c31563eb'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.048331 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1380/2722] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:46 2022] DELETE /identity/v3/users/551994f03add4f59b913b7e5c31563eb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:47.057914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:47.058487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:47.058487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:47.080102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:47.081712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:47.147739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1343/2723] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:47.158683 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:47.160208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:47.223653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fa817-d802-4f19-842c-a8106f5aa6da admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:47.227362 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:47.228192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:47.228501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:47.228778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:47.240230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:47.241729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:47.252373 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1381/2724] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:47.564350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:47.565884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:47.581049 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798585-8c44-41e1-b568-e9d2d1ec5228 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:47.587284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:47.588182 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:47.588462 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:47.588726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] PATH_INFO: `/v3/projects/186f5e50c9414335bbdae03ceff64ea8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:47.590097 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] RBAC: Authorizing `identity:delete_project(project_id=186f5e50c9414335bbdae03ceff64ea8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:47.592072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:47.629231 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:47.646538 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186f5e50c9414335bbdae03ceff64ea8', 'request_id': 'req-cbf67761-5840-43ea-b623-62db9d10467a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.653186 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186f5e50c9414335bbdae03ceff64ea8', 'request_id': 'req-cbf67761-5840-43ea-b623-62db9d10467a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.659437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186f5e50c9414335bbdae03ceff64ea8', 'request_id': 'req-cbf67761-5840-43ea-b623-62db9d10467a'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.674575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] The token cache is being invalidate because project 186f5e50c9414335bbdae03ceff64ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:47.674859 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.675782 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.677498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1344/2725] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:47 2022] DELETE /identity/v3/projects/186f5e50c9414335bbdae03ceff64ea8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:47.681924 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:47.692659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea4317-79a0-46fb-88f0-fdb7888056e2 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:47.755775 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:47.756571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:47.756844 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:47.757146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] PATH_INFO: `/v3/users/6c404a0e6dfc4d0e93ec67b5cb27f71a` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:47.758449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c404a0e6dfc4d0e93ec67b5cb27f71a)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:47.760387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:47.763531 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:47.822482 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c404a0e6dfc4d0e93ec67b5cb27f71a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.831283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c404a0e6dfc4d0e93ec67b5cb27f71a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.838969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c404a0e6dfc4d0e93ec67b5cb27f71a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.839597 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c404a0e6dfc4d0e93ec67b5cb27f71a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.840203 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-8ce8181f-6e6c-437a-8047-fe441958985e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c404a0e6dfc4d0e93ec67b5cb27f71a'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:47.859227 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1382/2726] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:47 2022] DELETE /identity/v3/users/6c404a0e6dfc4d0e93ec67b5cb27f71a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:48.141660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.143350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:48.164098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67761-5840-43ea-b623-62db9d10467a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.224790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.225691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.225971 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.226240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] PATH_INFO: `/v3/projects/04ddd6953ee0434a805b38b85b16693b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.227433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ddd6953ee0434a805b38b85b16693b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.229410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.278472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ddd6953ee0434a805b38b85b16693b', 'request_id': 'req-77f067a4-4c4e-41df-b11a-574621a80895'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:48.284502 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ddd6953ee0434a805b38b85b16693b', 'request_id': 'req-77f067a4-4c4e-41df-b11a-574621a80895'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:48.291018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ddd6953ee0434a805b38b85b16693b', 'request_id': 'req-77f067a4-4c4e-41df-b11a-574621a80895'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:48.306407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] The token cache is being invalidate because project 04ddd6953ee0434a805b38b85b16693b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:48.306690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:48.307326 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-77f067a4-4c4e-41df-b11a-574621a80895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:48.308883 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1345/2727] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:48 2022] DELETE /identity/v3/projects/04ddd6953ee0434a805b38b85b16693b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:48.314174 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.314719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.314913 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.333246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:48.334567 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:48.420397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1383/2728] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:48.431756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.432194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.432485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.451958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:48.453245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:48.465456 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1346/2729] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:48.467555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.477910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c296d5a-4e5a-4d16-9a9d-e84b9af118a7 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.482854 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.483514 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.483791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.484064 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.484853 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:48.485191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.486772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.499571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1384/2730] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:48.504365 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.514689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69d482-194a-4a25-a3d7-23509ba5df44 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.519440 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.520129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.520405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.520868 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.521604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.523773 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.559193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1347/2731] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:48 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:15:48.563759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.574803 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4539cd-f876-42d6-b671-1141e6148050 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.642864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.643581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.643880 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.644294 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.645420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.647613 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.664017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:48.670925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1385/2732] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:48 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:48.676124 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.690622 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ce11-0f8b-444c-921c-07df8456e0ff admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.696358 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.697072 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.700018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.700311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.701087 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.702704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.717653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1348/2733] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:48.721883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.732427 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61ad8-ce82-4636-9f72-1535c37d413c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.739145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.740165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.740486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.740832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.741643 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.743280 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a858d-83c0-480d-9eef-744782807d2f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.756377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1386/2734] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:48.760530 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.771507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59e23b-f4f9-4aa5-b0eb-411b56f87c75 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:48.776685 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:48.777607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.777936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.778541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] PATH_INFO: `/v3/projects/c8410704fa9c4679ab0188db2109ef71/users/6eb6ab7a15a6453cafb4965e196af9d8/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.780936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:48.798119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8410704fa9c4679ab0188db2109ef71, user_id=6eb6ab7a15a6453cafb4965e196af9d8, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:48.800940 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:48.815134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1349/2735] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:48 2022] PUT /identity/v3/projects/c8410704fa9c4679ab0188db2109ef71/users/6eb6ab7a15a6453cafb4965e196af9d8/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:48.826525 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:48.826818 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:48.827121 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:48.858183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] MFA Rules not processed for user `6eb6ab7a15a6453cafb4965e196af9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:48.859395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:48.934283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1387/2736] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:48.947369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:48.949557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.012310 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68cce-eabd-4483-ace6-599f678228a4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:49.022039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:49.022039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.022039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.022039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.027519 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:49.029493 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01cba5-ad26-4a1c-813f-3fef9e587ba5 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:49.041781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1350/2737] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:49.062726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:49.129088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97db6-cf25-4721-ae07-12d50a8fae92 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:49.134232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:49.135118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.135407 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.135680 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] PATH_INFO: `/v3/users/6eb6ab7a15a6453cafb4965e196af9d8` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.137048 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb6ab7a15a6453cafb4965e196af9d8)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:49.139063 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:49.194488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb6ab7a15a6453cafb4965e196af9d8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.204974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb6ab7a15a6453cafb4965e196af9d8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.212327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb6ab7a15a6453cafb4965e196af9d8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.212992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb6ab7a15a6453cafb4965e196af9d8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.213607 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb6ab7a15a6453cafb4965e196af9d8'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.225346 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:49.233447 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1388/2738] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:49 2022] DELETE /identity/v3/users/6eb6ab7a15a6453cafb4965e196af9d8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:49.238624 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.239129 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.239228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.256353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:49.257585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.318707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1351/2739] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:49.331998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:49.334343 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.395941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0526d-fb81-4d2e-9fe3-feca506cc431 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:49.399454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:49.400097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.400375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.400645 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.410644 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:49.412120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e9e95-bb35-4cdc-9ee8-f954ae8b7e1c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:49.422107 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1389/2740] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:49.715401 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:49.716957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.729224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95b8a4-1e5e-486c-b125-759e02d4e05b None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:49.735752 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:49.737739 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.737993 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.738304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] PATH_INFO: `/v3/projects/c8410704fa9c4679ab0188db2109ef71` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.739590 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8410704fa9c4679ab0188db2109ef71)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:49.741775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:49.789572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8410704fa9c4679ab0188db2109ef71', 'request_id': 'req-a4369a12-6902-44cc-8a6a-57914fcac02d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.796506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8410704fa9c4679ab0188db2109ef71', 'request_id': 'req-a4369a12-6902-44cc-8a6a-57914fcac02d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.802536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8410704fa9c4679ab0188db2109ef71', 'request_id': 'req-a4369a12-6902-44cc-8a6a-57914fcac02d'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.817110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] The token cache is being invalidate because project c8410704fa9c4679ab0188db2109ef71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:49.817399 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.818036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-a4369a12-6902-44cc-8a6a-57914fcac02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:49.819614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1352/2741] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:49 2022] DELETE /identity/v3/projects/c8410704fa9c4679ab0188db2109ef71 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:49.824720 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.825105 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.825483 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.844007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:49.845404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.910936 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1390/2742] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:49.912830 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.913249 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.913557 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.932492 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:49.934025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:49.942669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1353/2743] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:49.947017 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:49.958030 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fb44b-e58b-406d-8a59-824d7a588976 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:49.962923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:49.963579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:49.963893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:49.964172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:49.964929 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:49.965252 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:49.967249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:49.979207 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1391/2744] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:49.983563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:49.987972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:49.995322 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fa546-f92b-42f6-ae27-cd58f7cfffdd None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.002103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.002800 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.003095 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.003346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.004026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.006068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.033228 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1354/2745] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:49 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:50.037403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.048448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a8e-a072-4955-a7ef-06fb842e2887 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.107127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.107804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.108090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.108363 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.109081 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.111034 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.136575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1392/2746] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:50 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:50.140986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.152180 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e814a-cc14-4fd5-a0d8-2b75bdd8bdfd admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.158178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.158824 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.159103 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.159397 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.160125 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.161847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.172986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1355/2747] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:50.177449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.190040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08069982-ab02-4365-9b8c-4d5530a81dad admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.195029 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.195694 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.196045 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.196347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.197118 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.198734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48bd2-53e5-45f8-8b63-b8bd2cc75971 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.209765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1393/2748] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:50.213981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.224845 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9ed38-7834-4b88-affd-1f162fd0e1ae admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.230696 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.231592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.231871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.232273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] PATH_INFO: `/v3/projects/36602465a6dd42d4962e0c91252e3f54/users/e6e35b7c83fb4a0f9ab53d0e6c2c5524/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.248601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36602465a6dd42d4962e0c91252e3f54, user_id=e6e35b7c83fb4a0f9ab53d0e6c2c5524, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.251368 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.266808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1356/2749] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:50 2022] PUT /identity/v3/projects/36602465a6dd42d4962e0c91252e3f54/users/e6e35b7c83fb4a0f9ab53d0e6c2c5524/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:50.277120 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.277411 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.277657 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.285214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:50.306917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] MFA Rules not processed for user `e6e35b7c83fb4a0f9ab53d0e6c2c5524`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:50.308141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:50.382066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1394/2750] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:50.414363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.416453 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:50.477707 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c657-2738-4e71-8ee8-99559b4764b8 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.482289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.483065 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.483338 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.483632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.493041 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.494490 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d9fa2-8c52-4e6c-aa6b-839585ae3780 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.503347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1357/2751] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:50.518578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.590585 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a4298-3a88-4914-bdc5-6907af58347f None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.595703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.596614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.596890 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.597208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] PATH_INFO: `/v3/users/e6e35b7c83fb4a0f9ab53d0e6c2c5524` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.598370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e35b7c83fb4a0f9ab53d0e6c2c5524)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.600287 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.657281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e35b7c83fb4a0f9ab53d0e6c2c5524'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:50.666498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e35b7c83fb4a0f9ab53d0e6c2c5524'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:50.677334 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e35b7c83fb4a0f9ab53d0e6c2c5524'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:50.678433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e35b7c83fb4a0f9ab53d0e6c2c5524'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:50.679561 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e35b7c83fb4a0f9ab53d0e6c2c5524'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:50.700636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1395/2752] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:50 2022] DELETE /identity/v3/users/e6e35b7c83fb4a0f9ab53d0e6c2c5524 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:50.705983 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.706265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.706536 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.724996 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:50.726210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:50.786451 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1358/2753] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:50.796925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:50.798564 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:50.813915 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:50.863043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad68897-d98d-446f-b4f4-9ce3a83878fd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:50.866867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:50.867511 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:50.867797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:50.868077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:50.877746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:50.879204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf98fdc-34b9-4219-b131-8e3307cd6dcb service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:50.889101 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1396/2754] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:51.179506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.181098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:51.194743 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f79d-6ee8-4f0e-a1e5-4d0604079bfe None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.199683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.200736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.201807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.202102 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] PATH_INFO: `/v3/projects/36602465a6dd42d4962e0c91252e3f54` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.203415 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36602465a6dd42d4962e0c91252e3f54)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.205700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.251132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:51.256016 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36602465a6dd42d4962e0c91252e3f54', 'request_id': 'req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:51.263505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36602465a6dd42d4962e0c91252e3f54', 'request_id': 'req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:51.269204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36602465a6dd42d4962e0c91252e3f54', 'request_id': 'req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:51.284289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] The token cache is being invalidate because project 36602465a6dd42d4962e0c91252e3f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:51.284676 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:51.285790 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8b1d88d4-17a6-4b30-9c6f-ab47a1df2d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:51.288143 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1359/2755] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:51 2022] DELETE /identity/v3/projects/36602465a6dd42d4962e0c91252e3f54 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:51.294049 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.294614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.294895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.313700 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:51.314996 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:51.381944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1397/2756] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:51.390911 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.391202 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.391478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.412266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:51.413508 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:51.424784 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1360/2757] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:51.429707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.440246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4dc428-5fef-46c6-9df6-5b70b88a8176 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.445238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.445899 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.446180 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.446456 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.447195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:51.447505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.449151 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.461267 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1398/2758] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:51.465649 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.475565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494ed6-4b51-451b-ac21-d00e3b55e046 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.482039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.482706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.482968 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.483234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.483914 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.485882 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.518488 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1361/2759] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:51.519640 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.530040 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b99e-305e-4445-a112-c682239bf08b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.591670 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.592350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.592634 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.592933 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.593630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.595574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.619043 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1399/2760] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:51.620361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.630994 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656eb5d-dde9-4979-8edb-b610f1cf49e3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.636573 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.637323 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.637599 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.637942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.638890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.640506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.651960 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1362/2761] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:51.655981 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.666916 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca36d-9f0f-4fde-befc-a9fb202a703c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.672811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.673510 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.673793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.674066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.674868 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.676415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b168a1-a139-4a4d-b4eb-7c876fde5249 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.687170 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1400/2762] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:51.692044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.708157 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1931f3-884f-4461-9eb7-a3985e88d7cb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.716615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.717559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.717840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.718115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] PATH_INFO: `/v3/projects/66b40c298d354788b7feb2257abd6396/users/4b164f365c0a47ab99a167c353322810/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.733669 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b40c298d354788b7feb2257abd6396, user_id=4b164f365c0a47ab99a167c353322810, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.736325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.752068 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1363/2763] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:51 2022] PUT /identity/v3/projects/66b40c298d354788b7feb2257abd6396/users/4b164f365c0a47ab99a167c353322810/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:51.762684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.763243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.763507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.791181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] MFA Rules not processed for user `4b164f365c0a47ab99a167c353322810`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:51.792397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:51.803022 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:51.861236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1401/2764] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:51.866709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.933254 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ccf87-3dd6-4243-a693-27664cd7dd7e admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:51.938325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:51.938984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:51.939284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:51.939564 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:51.940242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:51.942175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:51.974190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1364/2765] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:51.975409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:51.986685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f085322-7a59-41fe-8d55-2012f5dc0489 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.047807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.048485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.048766 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.049084 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.049779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.051998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.075164 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1402/2766] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:51 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:52.079391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:52.090693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41d7b-18e7-47ab-93c9-4c9efcf54400 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.095737 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.096396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.096680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.096981 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.097711 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.099321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.110183 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1365/2767] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:52.114485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:52.125431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddf834-f744-437b-8390-7ab0c59e4093 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.131316 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.132212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.132501 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.132778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] PATH_INFO: `/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1/users/c84232e69d4f46378f4e78846d4531f9/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.156241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5956aec4e54841aeb1b9e8025b31b3b1, user_id=c84232e69d4f46378f4e78846d4531f9, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.158965 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.176210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1403/2768] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:52 2022] PUT /identity/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1/users/c84232e69d4f46378f4e78846d4531f9/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:52.180520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:52.192683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582682-773d-47cf-b872-11b3c5162df7 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.233840 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:52.253941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.254626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.254880 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.255151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.255879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.257529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e286f-cb3a-4f13-9c31-af2e86c027eb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.267426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1366/2769] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:15:52.271723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:52.282350 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9db38a-2039-47a1-b5e4-8fe813acf984 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.287455 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.288341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.288619 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.288866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] PATH_INFO: `/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1/users/c84232e69d4f46378f4e78846d4531f9/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.290952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5956aec4e54841aeb1b9e8025b31b3b1, user_id=c84232e69d4f46378f4e78846d4531f9, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.293636 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.303887 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1404/2770] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:52 2022] PUT /identity/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1/users/c84232e69d4f46378f4e78846d4531f9/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:15:52.314177 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.314474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.314780 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.344378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] MFA Rules not processed for user `c84232e69d4f46378f4e78846d4531f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:52.345663 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:52.421700 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1367/2771] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:52.433296 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:52.435653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:52.505157 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748d38-7ff7-471a-9314-c156140c9243 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:52.509284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:52.509939 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:52.510219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:52.510492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:52.528954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:52.576347 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:52.577831 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26a86d-2939-4a02-8728-b4e5200ebfa0 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:52.587954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1405/2772] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:56.216035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:56.282269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8e5e8-d7a2-4190-a6de-c78c17cb64aa None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:56.287648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:56.288537 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:56.288822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:56.289126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] PATH_INFO: `/v3/users/4b164f365c0a47ab99a167c353322810` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:56.290371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b164f365c0a47ab99a167c353322810)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:56.292335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:56.344610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b164f365c0a47ab99a167c353322810'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.351946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b164f365c0a47ab99a167c353322810'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.360314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b164f365c0a47ab99a167c353322810'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.361000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b164f365c0a47ab99a167c353322810'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.361652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b164f365c0a47ab99a167c353322810'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.363544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:56.381583 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1368/2773] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:56 2022] DELETE /identity/v3/users/4b164f365c0a47ab99a167c353322810 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:56.387175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:56.388091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:56.388185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:56.406708 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:56.407953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:56.471486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1406/2774] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:56.481524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:56.483581 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:56.556789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bda7-aca6-44ac-9afa-db73b1bdb392 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:56.563432 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:56.564385 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:56.564759 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:56.565157 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:56.578888 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:56.580697 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:56.590841 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1369/2775] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:56.891788 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:56.893284 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:56.907826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fb227-5df2-485d-b23b-732e25be03e2 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:56.913136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:56.914696 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:56.915165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:56.915444 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] PATH_INFO: `/v3/projects/66b40c298d354788b7feb2257abd6396` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:56.916870 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b40c298d354788b7feb2257abd6396)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:56.918726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:56.970729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b40c298d354788b7feb2257abd6396', 'request_id': 'req-18d4d843-3307-4b99-9da4-0ba30f681bc0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.977116 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b40c298d354788b7feb2257abd6396', 'request_id': 'req-18d4d843-3307-4b99-9da4-0ba30f681bc0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.983021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b40c298d354788b7feb2257abd6396', 'request_id': 'req-18d4d843-3307-4b99-9da4-0ba30f681bc0'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.996449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] The token cache is being invalidate because project 66b40c298d354788b7feb2257abd6396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:56.996729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.997595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:56.999345 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1407/2776] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:56 2022] DELETE /identity/v3/projects/66b40c298d354788b7feb2257abd6396 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:57.003430 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:57.013194 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee66e-b541-46e1-9f8b-0e48d7bcab5d service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:57.072005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:57.072890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.073203 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.073471 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] PATH_INFO: `/v3/users/c84232e69d4f46378f4e78846d4531f9` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.074712 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84232e69d4f46378f4e78846d4531f9)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:57.076783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:57.132989 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84232e69d4f46378f4e78846d4531f9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.141962 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84232e69d4f46378f4e78846d4531f9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.150811 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84232e69d4f46378f4e78846d4531f9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.151496 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84232e69d4f46378f4e78846d4531f9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.152107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-8278a21c-a3d8-4731-82bb-2a4eba82ee55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84232e69d4f46378f4e78846d4531f9'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.169972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1370/2777] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:15:57 2022] DELETE /identity/v3/users/c84232e69d4f46378f4e78846d4531f9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:57.453669 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:57.455746 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:57.470028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d843-3307-4b99-9da4-0ba30f681bc0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:57.511674 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:57.531143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:57.532092 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.532473 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.532740 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] PATH_INFO: `/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.534707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=5956aec4e54841aeb1b9e8025b31b3b1)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:57.537515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:57.594600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5956aec4e54841aeb1b9e8025b31b3b1', 'request_id': 'req-3f0465d3-422a-4d5f-99c7-fed7f093b8df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.600801 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5956aec4e54841aeb1b9e8025b31b3b1', 'request_id': 'req-3f0465d3-422a-4d5f-99c7-fed7f093b8df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.606748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5956aec4e54841aeb1b9e8025b31b3b1', 'request_id': 'req-3f0465d3-422a-4d5f-99c7-fed7f093b8df'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.622524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] The token cache is being invalidate because project 5956aec4e54841aeb1b9e8025b31b3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:15:57.622806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.623432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-3f0465d3-422a-4d5f-99c7-fed7f093b8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:15:57.625214 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1408/2778] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:15:57 2022] DELETE /identity/v3/projects/5956aec4e54841aeb1b9e8025b31b3b1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:57.630378 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.630867 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.630986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.638781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:57.650789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:57.652545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:57.714053 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1371/2779] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:57.779062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.779449 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.780171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.803416 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:57.804555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:57.814047 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1409/2780] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:57.818521 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:57.829280 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce5f83-cf2a-4dbe-9ca9-8f088ffca2c1 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:57.835610 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:57.836287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.836566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.836838 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.837601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:15:57.837907 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:57.839522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:57.852895 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1372/2781] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:15:57.858920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:57.868664 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abf77d-8d65-4da5-be39-42fce249dd2c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:57.875453 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:57.876112 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.876389 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.876661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.877386 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:57.879312 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:57.909396 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1410/2782] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:15:57 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:57.913906 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:57.924465 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99e477-967c-4f7a-9c8e-ee6b2a8ee46b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:57.980576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:57.981279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:57.981563 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:57.981837 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:57.982601 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:57.984621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:58.006085 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1373/2783] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:15:57 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:15:58.010508 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:58.021275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641aa6f-bc19-415c-bbe7-b62697b64848 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:58.026922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:58.027589 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.027872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.028143 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.028877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:58.030546 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:58.041472 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1411/2784] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:58.045683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:58.056391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3159490-17f2-4026-8ef8-902ec1fec6ed admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:58.062005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:58.062672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.062952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.063220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.063943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:58.065572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfd406-c352-4731-8185-ac332eedce0c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:58.075988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1374/2785] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:15:58.080922 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:58.091435 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dff84-af05-4f0f-af76-501f0562e95b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:58.097796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:58.098688 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.098963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.099232 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] PATH_INFO: `/v3/projects/1affbea314ef42ae9c8eb051f633e3df/users/f0c23febe0064a6abddc75cae27e7ef4/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.116352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1affbea314ef42ae9c8eb051f633e3df, user_id=f0c23febe0064a6abddc75cae27e7ef4, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:58.119058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:58.137605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1412/2786] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:15:58 2022] PUT /identity/v3/projects/1affbea314ef42ae9c8eb051f633e3df/users/f0c23febe0064a6abddc75cae27e7ef4/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:15:58.149849 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeec3b6-d088-41ea-83cc-b93446c5441b None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.150281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeec3b6-d088-41ea-83cc-b93446c5441b None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.150569 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeec3b6-d088-41ea-83cc-b93446c5441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.171134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7aeec3b6-d088-41ea-83cc-b93446c5441b None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:58.172337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7aeec3b6-d088-41ea-83cc-b93446c5441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:58.241565 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1375/2787] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:58.251686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:58.253571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:58.315895 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc64a45-fbfc-495f-9c4a-f13582684fec admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:58.320033 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:58.320684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.320993 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.321297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.331462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:58.332894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:58.344717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1413/2788] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:15:58.970862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6234ad-322f-44fa-96fe-565dc1529d2a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:58.971417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6234ad-322f-44fa-96fe-565dc1529d2a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:58.971417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6234ad-322f-44fa-96fe-565dc1529d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:58.992005 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:59.000281 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-ea6234ad-322f-44fa-96fe-565dc1529d2a None None] MFA Rules not processed for user `f0c23febe0064a6abddc75cae27e7ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:15:59.002392 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-ea6234ad-322f-44fa-96fe-565dc1529d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:59.071104 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1376/2789] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:15:59.082974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:15:59.084384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:15:59.097917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:15:59.144268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb18483-84c5-44ec-a614-b0b7b595e2c8 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:15:59.148169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:15:59.148830 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:15:59.149144 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:15:59.149420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:15:59.158265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:15:59.159709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:15:59.168631 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1414/2790] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:15:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:01.592662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:01.593592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:01.593592 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:01.617532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] MFA Rules not processed for user `f0c23febe0064a6abddc75cae27e7ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:16:01.618801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:01.626862 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1377/2791] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:16:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:16:01.883641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:01.885872 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:01.901559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504c5ee-b1c2-4f7c-b535-2eaccf1ff830 service cinder] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:01.907169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:01.908007 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:01.908415 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:01.908807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:01.911028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:01.922445 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:01.923955 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:01.933538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1415/2792] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:02.253648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:02.255389 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:02.333218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6261337-b31a-4841-8462-c26b9b58fc22 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:02.338155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:02.338810 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:02.339091 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:02.339361 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:02.341219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:02.353287 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:02.354917 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f6beb-a744-4d64-bb8b-5082aba8224e service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:02.364997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1378/2793] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:11.929176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=27 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:12.360952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=62 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:13.141864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:13.143392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:13.226056 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19129b-664a-495e-acda-e427e0aba42a service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:13.231430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:13.232090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:13.232375 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:13.232647 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:13.234462 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:13.246845 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:13.248436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:13.259204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1416/2794] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:21.185672 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:21.186528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:21.186528 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:21.203892 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] MFA Rules not processed for user `f0c23febe0064a6abddc75cae27e7ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:16:21.205691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:21.214026 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1379/2795] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:16:23.254935 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=34 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:23.794404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:23.805970 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75627d2b-0441-4da5-a37f-fbd4b4868d5c service nova] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:23.809666 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:23.810330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:23.810612 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:23.810883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:23.813356 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:23.884552 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:23.886124 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ec828-b562-4bad-a4fe-d6d71aebcca3 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:23.896167 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1417/2796] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:16:25.892147 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:25.893844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:25.913149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e612-bd1e-4acc-ad1d-987757b196ea None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:25.918928 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:25.919548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:25.919909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:25.920271 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:25.922419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:25.935554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:25.937354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f09bb-18a6-4675-a49f-70936ce417fe service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:25.949614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1380/2797] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:33.891442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=30 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:35.945604 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=9 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:39.490632 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:39.491165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:39.491426 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:39.510098 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] MFA Rules not processed for user `f2f9826f53b44e1bbaf721d66de1356b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:16:39.511419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:39.577849 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1418/2798] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:16:46.015155 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:46.015701 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:46.015986 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:46.046756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:16:46.049052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:46.114038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1381/2799] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Jun 28 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:16:46.174902 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:46.176454 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:46.258686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0954-31c4-4dfd-bf5a-8fdf92ccf919 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:46.266257 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:46.266616 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:46.266900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:46.267175 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:46.281442 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:46.283096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:46.294065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1419/2800] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:16:48.725559 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:16:48.737425 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89dcf8-022e-41e1-97f6-63c4388a319a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:16:48.741523 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:16:48.742171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:16:48.742427 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:16:48.742858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:16:48.744891 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:16:48.845306 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:16:48.847148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6bfbb-5b76-45b5-998b-dd36f87551f7 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:16:48.868460 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1382/2801] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:16:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:16:56.290434 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=57 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:16:58.865951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=66 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:17:03.262832 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:03.303488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d072c-703f-40b2-a368-a3018ba73f5b service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:03.308538 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:03.309238 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:03.309505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:03.309786 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:03.311685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:03.381599 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:03.383058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:03.407093 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1420/2802] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:17:03.777084 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:03.777615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:03.777726 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:03.797858 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] MFA Rules not processed for user `81b68417e9e043b3abc779986564daff`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:17:03.799120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:03.807258 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1383/2803] 10.222.0.55 () {58 vars in 967 bytes} [Tue Jun 28 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:17:03.820001 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:03.832980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f33afc-90f1-4c37-becf-e7fbcc93ffa8 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:03.838967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee9dc1ba0084428c9da313ebda0458e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:03.839622 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:03.839912 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:03.840176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:03.848726 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:03.850366 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:03.858734 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1421/2804] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Jun 28 12:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 28 12:17:05.899725 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:05.918199 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff34794-0c9d-4207-82ea-cc68533ff0f5 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:05.923793 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:05.925138 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:05.925637 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:05.926120 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] PATH_INFO: `/v3/users/f0c23febe0064a6abddc75cae27e7ef4` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:05.927607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c23febe0064a6abddc75cae27e7ef4)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:05.930052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:05.990205 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c23febe0064a6abddc75cae27e7ef4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.001437 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c23febe0064a6abddc75cae27e7ef4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.015699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c23febe0064a6abddc75cae27e7ef4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.016379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c23febe0064a6abddc75cae27e7ef4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.017175 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c23febe0064a6abddc75cae27e7ef4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.039751 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1384/2805] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:17:05 2022] DELETE /identity/v3/users/f0c23febe0064a6abddc75cae27e7ef4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:06.308249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:06.322269 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007a932-15f7-401f-b4a1-9af961e9ae1c service swift] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:06.384236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:06.385085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:06.385398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:06.385673 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] PATH_INFO: `/v3/projects/1affbea314ef42ae9c8eb051f633e3df` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:06.386920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1affbea314ef42ae9c8eb051f633e3df)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:06.388863 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:06.452253 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1affbea314ef42ae9c8eb051f633e3df', 'request_id': 'req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.454733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:17:06.459963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1affbea314ef42ae9c8eb051f633e3df', 'request_id': 'req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.467223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1affbea314ef42ae9c8eb051f633e3df', 'request_id': 'req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.483130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] The token cache is being invalidate because project 1affbea314ef42ae9c8eb051f633e3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:17:06.483352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.484179 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:06.485875 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1422/2806] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:17:06 2022] DELETE /identity/v3/projects/1affbea314ef42ae9c8eb051f633e3df => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:09.886024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:09.887679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:09.903334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011e223-0da1-4ebe-80cc-b2a789f15aed admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:09.967534 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:09.968429 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:09.968706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:09.969012 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] PATH_INFO: `/v3/users/9f56095d6bac4f108a73790ac9004f3b` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:09.970366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f56095d6bac4f108a73790ac9004f3b)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:09.972383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:10.019889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f56095d6bac4f108a73790ac9004f3b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.026896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f56095d6bac4f108a73790ac9004f3b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.035145 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f56095d6bac4f108a73790ac9004f3b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.035881 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f56095d6bac4f108a73790ac9004f3b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.036489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f56095d6bac4f108a73790ac9004f3b'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.041082 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:17:10.057045 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1385/2807] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:17:09 2022] DELETE /identity/v3/users/9f56095d6bac4f108a73790ac9004f3b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:10.236604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:10.238198 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:10.251893 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1c19-ee2d-42ae-a316-0c2ec81b587b admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:10.315181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:10.316065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:10.316352 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:10.316627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] PATH_INFO: `/v3/projects/7235a9c7eff24e0e94900ec2abc92ae5` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:10.317910 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7235a9c7eff24e0e94900ec2abc92ae5)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:10.319876 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:10.370328 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7235a9c7eff24e0e94900ec2abc92ae5', 'request_id': 'req-7e2194de-628a-45ec-a4f2-005cedbbb6a5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.376600 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7235a9c7eff24e0e94900ec2abc92ae5', 'request_id': 'req-7e2194de-628a-45ec-a4f2-005cedbbb6a5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.382926 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7235a9c7eff24e0e94900ec2abc92ae5', 'request_id': 'req-7e2194de-628a-45ec-a4f2-005cedbbb6a5'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.397430 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] The token cache is being invalidate because project 7235a9c7eff24e0e94900ec2abc92ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:17:10.397719 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.398342 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.400243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1423/2808] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:17:10 2022] DELETE /identity/v3/projects/7235a9c7eff24e0e94900ec2abc92ae5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:10.404134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:10.405682 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:10.419467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3466e77-8dcf-4ddf-bac3-975e7ddaca8b admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:10.480353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:10.481269 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:10.481542 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:10.481813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] PATH_INFO: `/v3/users/771d190d4a174b139499028e3a028292` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:10.495410 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=771d190d4a174b139499028e3a028292)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:10.497395 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:10.551674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771d190d4a174b139499028e3a028292'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.559004 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771d190d4a174b139499028e3a028292'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.569893 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771d190d4a174b139499028e3a028292'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.570506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771d190d4a174b139499028e3a028292'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.571208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771d190d4a174b139499028e3a028292'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.595716 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1386/2809] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:17:10 2022] DELETE /identity/v3/users/771d190d4a174b139499028e3a028292 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:10.785547 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:17:10.787100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:17:10.802615 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2194de-628a-45ec-a4f2-005cedbbb6a5 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:17:10.861274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:17:10.862149 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:17:10.862428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:17:10.862699 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] PATH_INFO: `/v3/projects/a0d938c674cb43faae751b86e5ce54a5` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:17:10.873879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d938c674cb43faae751b86e5ce54a5)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:17:10.875824 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:17:10.878461 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:17:10.926178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d938c674cb43faae751b86e5ce54a5', 'request_id': 'req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.932605 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d938c674cb43faae751b86e5ce54a5', 'request_id': 'req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.938718 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d938c674cb43faae751b86e5ce54a5', 'request_id': 'req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.954592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] The token cache is being invalidate because project a0d938c674cb43faae751b86e5ce54a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:17:10.954879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.955553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e1e3fd9b-4a11-4ba2-851e-94a3897d9221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:17:10.957224 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1424/2810] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:17:10 2022] DELETE /identity/v3/projects/a0d938c674cb43faae751b86e5ce54a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:17:20.590951 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=12,keystone:SELECT=63,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:17:20.951377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:18:18.100141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:18:18.101576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:18.198000 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00f022-d502-4bf4-9458-09f17f4ec053 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:18:18.261260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:18:18.261930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:18:18.262208 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:18:18.262481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:18:18.264339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:18.275808 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:18:18.277458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:18:18.287613 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1387/2811] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:18:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:18:27.505623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:18:27.506204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:18:27.506204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:18:27.524659 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] MFA Rules not processed for user `f2f9826f53b44e1bbaf721d66de1356b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:18:27.525928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:27.584243 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1425/2812] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:18:28.283141 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=67 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:18:37.576823 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=25 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:18:57.595969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:18:57.597576 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:57.681595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29190a-1c37-4cb5-ba6f-c3651b5755e6 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:18:57.685442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:18:57.686101 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:18:57.686376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:18:57.686645 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:18:57.688472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:57.701363 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:18:57.702809 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:18:57.713674 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1388/2813] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:18:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:18:58.823387 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:18:58.834628 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef096-5f71-45e6-84eb-1df6421c4836 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:18:58.838653 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:18:58.839321 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:18:58.839598 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:18:58.839867 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:18:58.841911 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:18:58.925681 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:18:58.927196 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:18:58.938420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1426/2814] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:18:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:19:07.707781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=31 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:19:08.934184 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=37 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:19:44.931035 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:19:44.933152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:44.980025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e8bbe-707d-48ab-b3cf-2bd65d9600e1 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:19:44.987706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:19:44.988351 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:19:44.988621 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:19:44.988879 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:19:44.991017 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:45.002552 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:19:45.004142 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:19:45.013839 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1389/2815] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:19:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:19:46.438165 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:19:46.439722 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:46.522204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a98d-5d24-46e6-83f0-c3994cfdbe5c service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:19:46.528288 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:19:46.529518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:19:46.529798 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:19:46.530091 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:19:46.533398 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:46.552382 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:19:46.553864 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850abd-af03-49ed-a17b-99d0650cf203 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:19:46.565524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1427/2816] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:19:54.441157 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:19:54.443458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:54.463059 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f3b29-3b82-4b01-aebe-f1cb47023c06 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:19:54.469589 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:19:54.470242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:19:54.470524 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:19:54.470796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:19:54.472935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:19:54.486801 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:19:54.488245 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:19:54.499549 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1390/2817] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:19:56.560235 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=35 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:02.792246 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:02.792892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:02.792892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:02.810335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] MFA Rules not processed for user `f2f9826f53b44e1bbaf721d66de1356b`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:20:02.812271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:02.819528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1428/2818] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:20:04.494396 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=21 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:05.257334 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:05.258792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:05.271211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46707e5e-9fe4-4d5e-9257-d60bc7567c88 service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:05.275060 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:05.275721 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:05.275997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:05.276407 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:05.278246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:05.289158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:05.290626 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:05.299506 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1391/2819] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:12.805195 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:15.295498 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=6 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:44.910999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:44.928340 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df5c6e-887a-48d9-b292-e1249ad30ebe None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:44.934917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:44.935571 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:44.935860 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:44.936129 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:44.938575 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:45.006466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:45.007920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:45.017591 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1429/2820] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:46.142974 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:46.144930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:46.164121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86659dbc-2665-498c-bf65-7ceadb24d0f1 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:46.170909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77cc74fa2a6c48d0b85e3b2e70f451f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:46.171572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:46.171848 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:46.172115 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:46.174118 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:46.188341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:46.190014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:46.202921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1392/2821] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:50.039556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:50.048744 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d9e94-f320-4cb8-bd49-476dc6731faf service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:50.053367 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:50.054026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:50.054310 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:50.054580 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:50.056587 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:50.125141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:50.126573 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:50.136409 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1430/2822] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:53.016505 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:53.020211 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:53.047304 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06995477-f618-4725-bdca-49d7e83bef23 service nova] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:53.061479 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:53.062371 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:53.062548 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:53.062709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:53.068972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:53.081958 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:53.084193 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:53.101820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1393/2823] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:56.240779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:56.270133 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb6b58-4184-447d-b90e-39046676156e service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:56.283279 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:56.283999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:56.284281 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:56.284557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:56.286495 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:56.303463 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:56.306069 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:56.312080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:56.314999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:56.321765 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1431/2824] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:20:56.335337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0525af-5bc3-46e4-9bc8-c116a89a3ff4 service placement] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:56.354820 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:56.355755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:56.356039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:56.356314 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] PATH_INFO: `/v3/users/1c0c3b5771de4561835b6bb4a74c86ed` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:56.358501 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0c3b5771de4561835b6bb4a74c86ed)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:56.360640 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:56.408654 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0c3b5771de4561835b6bb4a74c86ed'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.415934 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0c3b5771de4561835b6bb4a74c86ed'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.424944 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0c3b5771de4561835b6bb4a74c86ed'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.425544 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0c3b5771de4561835b6bb4a74c86ed'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.426170 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0c3b5771de4561835b6bb4a74c86ed'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.443982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1394/2825] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:20:56 2022] DELETE /identity/v3/users/1c0c3b5771de4561835b6bb4a74c86ed => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:20:56.623392 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:56.635223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e481d-e7c3-40c0-9121-1548ae714b2c service placement] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:56.693384 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:56.694271 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:56.694553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:56.694829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] PATH_INFO: `/v3/projects/56f6e9f5c280479d9050a6bfdde0d68b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:56.696054 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f6e9f5c280479d9050a6bfdde0d68b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:56.698086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:56.721923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=100 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:56.745155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f6e9f5c280479d9050a6bfdde0d68b', 'request_id': 'req-c310987f-9f3d-420d-84df-28d8615484bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.751747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f6e9f5c280479d9050a6bfdde0d68b', 'request_id': 'req-c310987f-9f3d-420d-84df-28d8615484bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.757623 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f6e9f5c280479d9050a6bfdde0d68b', 'request_id': 'req-c310987f-9f3d-420d-84df-28d8615484bc'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.771639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] The token cache is being invalidate because project 56f6e9f5c280479d9050a6bfdde0d68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:20:56.771987 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.772527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.774140 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1432/2826] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:20:56 2022] DELETE /identity/v3/projects/56f6e9f5c280479d9050a6bfdde0d68b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:20:56.778264 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:56.779909 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:56.793579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c3ca-1cad-43d3-9770-a44a662919a5 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:56.853558 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:56.854366 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:56.854648 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:56.854923 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] PATH_INFO: `/v3/users/a3a5af42b5914ab9a0b96b2238f859ea` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:56.856185 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a5af42b5914ab9a0b96b2238f859ea)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:56.858263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:56.918039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a5af42b5914ab9a0b96b2238f859ea'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.919646 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:20:56.928627 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a5af42b5914ab9a0b96b2238f859ea'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.937307 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a5af42b5914ab9a0b96b2238f859ea'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.937952 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a5af42b5914ab9a0b96b2238f859ea'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.938762 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a5af42b5914ab9a0b96b2238f859ea'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:56.956503 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1395/2827] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:20:56 2022] DELETE /identity/v3/users/a3a5af42b5914ab9a0b96b2238f859ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:20:57.137941 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:20:57.139498 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:20:57.153065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310987f-9f3d-420d-84df-28d8615484bc admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:20:57.213539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:20:57.214420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:20:57.214724 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:20:57.214998 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] PATH_INFO: `/v3/projects/bfb4f296fbb940d3a137a41a0d340f59` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:20:57.216230 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb4f296fbb940d3a137a41a0d340f59)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:20:57.218222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:20:57.270177 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb4f296fbb940d3a137a41a0d340f59', 'request_id': 'req-9707d812-5697-47f8-8431-a8b9829e1519'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:57.276263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb4f296fbb940d3a137a41a0d340f59', 'request_id': 'req-9707d812-5697-47f8-8431-a8b9829e1519'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:57.282957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb4f296fbb940d3a137a41a0d340f59', 'request_id': 'req-9707d812-5697-47f8-8431-a8b9829e1519'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:57.296225 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] The token cache is being invalidate because project bfb4f296fbb940d3a137a41a0d340f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:20:57.296419 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:57.300437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:20:57.300437 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1433/2828] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:20:57 2022] DELETE /identity/v3/projects/bfb4f296fbb940d3a137a41a0d340f59 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:06.953132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=12,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:07.293404 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:38.983458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:38.984946 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:39.046930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf6c7c-ebf0-48e1-a4e2-c1b8bc090982 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:39.101126 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:39.101775 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:39.102052 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:39.102320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:39.104277 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:39.185825 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:39.187288 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:39.198339 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1396/2829] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:21:42.468953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:42.470560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:42.483709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d812-5697-47f8-8431-a8b9829e1519 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:42.488339 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:42.489276 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:42.489556 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:42.489828 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] PATH_INFO: `/v3/users/f2f9826f53b44e1bbaf721d66de1356b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:42.491010 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f9826f53b44e1bbaf721d66de1356b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:42.492952 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:42.542431 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f9826f53b44e1bbaf721d66de1356b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.551816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f9826f53b44e1bbaf721d66de1356b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.559989 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f9826f53b44e1bbaf721d66de1356b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.560794 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f9826f53b44e1bbaf721d66de1356b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.561539 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-b2d8266f-232b-4136-abac-247b63114ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f9826f53b44e1bbaf721d66de1356b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.580760 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1434/2830] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:21:42 2022] DELETE /identity/v3/users/f2f9826f53b44e1bbaf721d66de1356b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:42.765077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:42.780826 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903821f-49dd-48e1-8eda-0a0f41440c85 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:42.843025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:42.844300 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:42.844617 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:42.844896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] PATH_INFO: `/v3/projects/6cf87482b69e40dda5b8104e3e1b12e8` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:42.846196 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf87482b69e40dda5b8104e3e1b12e8)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:42.848109 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:42.856699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=100 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:42.897353 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf87482b69e40dda5b8104e3e1b12e8', 'request_id': 'req-16534f27-1c43-439a-913e-35b00df96e94'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.903486 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf87482b69e40dda5b8104e3e1b12e8', 'request_id': 'req-16534f27-1c43-439a-913e-35b00df96e94'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.909154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf87482b69e40dda5b8104e3e1b12e8', 'request_id': 'req-16534f27-1c43-439a-913e-35b00df96e94'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.922266 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] The token cache is being invalidate because project 6cf87482b69e40dda5b8104e3e1b12e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:21:42.922556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.923152 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-16534f27-1c43-439a-913e-35b00df96e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:42.924699 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1397/2831] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:21:42 2022] DELETE /identity/v3/projects/6cf87482b69e40dda5b8104e3e1b12e8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:42.929855 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:42.930145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:42.930550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:42.992186 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:42.993436 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:43.060728 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1435/2832] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:43.068822 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.069132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.069405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.088575 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:43.089812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:43.099105 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1398/2833] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:43.103488 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.114025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabb8fb-ad27-4546-80b1-acf17f767226 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.119764 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.120464 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.120747 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.121072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.121809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:21:43.122138 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.123892 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.136283 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1436/2834] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:21:43.141756 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.151367 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577f350-fb34-4650-acdb-94c8873a8d74 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.157139 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.157814 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.158088 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.158354 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.159024 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.160936 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.209595 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1399/2835] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:21:43.213908 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.224536 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832d8c-cc06-450b-88f4-4f460f08f448 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.253814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:43.288843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.289569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.289850 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.290185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.290891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.292834 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.315771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1437/2836] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:21:43.320013 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.331355 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417885f-d6ee-40f5-838e-86f9bb68702a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.336331 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.337028 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.337309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.337615 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.338341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.339948 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.350805 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1400/2837] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:43.354661 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.365073 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49f7bf-d098-44f9-8418-138276812f3f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.369793 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.370448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.370725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.370967 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.371684 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.373553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de04aa-9fad-4a51-9a97-1cd4f8ac2a11 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.383371 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1438/2838] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:43.387426 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.397959 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b358d-a000-4a29-9454-d43722ae7196 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.403777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.404736 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.405047 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.405289 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] PATH_INFO: `/v3/projects/a6dc77d70190462e93932af4f76c2dac/users/882258768cfb43da96fe1d896d415894/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.421680 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc77d70190462e93932af4f76c2dac, user_id=882258768cfb43da96fe1d896d415894, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.424383 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.440357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1401/2839] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:43 2022] PUT /identity/v3/projects/a6dc77d70190462e93932af4f76c2dac/users/882258768cfb43da96fe1d896d415894/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:43.452410 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.452716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.453032 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.479679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] MFA Rules not processed for user `882258768cfb43da96fe1d896d415894`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:43.480874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:43.560712 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1439/2840] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:43.565584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.637995 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcecd3-ef1a-4047-a5ae-6de9e90e41ed admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.643463 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.644123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.644404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.644679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.645403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.647342 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.674935 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1402/2841] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:21:43.679142 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.691641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbeaed-af11-45b7-8007-b8289ab8f045 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.755192 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.755866 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.756145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.756418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.757123 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.759062 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.783583 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1440/2842] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:21:43 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:21:43.787190 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.798748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a2103-9211-4783-b261-8af1881367f3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.804449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.805394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.805688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.805957 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.806671 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.808292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.810609 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:43.821379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1403/2843] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:43.826239 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.837006 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57c74-2144-4eac-90c4-0ebdb2eb830c admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.842524 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.843418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.843706 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.843983 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] PATH_INFO: `/v3/projects/2fa6816d738140b2a9451a0176722cf4/users/7d8b90b0da32476fb0a36b69348bed93/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.860588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa6816d738140b2a9451a0176722cf4, user_id=7d8b90b0da32476fb0a36b69348bed93, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.863357 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.890603 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1441/2844] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:43 2022] PUT /identity/v3/projects/2fa6816d738140b2a9451a0176722cf4/users/7d8b90b0da32476fb0a36b69348bed93/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:21:43.894921 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.905644 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09507089-a8b0-4b05-a19d-198238884b46 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.963470 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:43.964134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:43.964404 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:43.964677 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:43.965441 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:43.967094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673936b0-4e90-47f8-8f4e-4ac020f68a4b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:43.977695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1404/2845] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:43.982299 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:43.987811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:43.994368 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998548-5c09-46d8-a237-29fd608fb3cd admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:43.999742 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:44.000639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.000950 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.001234 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] PATH_INFO: `/v3/projects/2fa6816d738140b2a9451a0176722cf4/users/7d8b90b0da32476fb0a36b69348bed93/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.003183 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa6816d738140b2a9451a0176722cf4, user_id=7d8b90b0da32476fb0a36b69348bed93, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:44.005920 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:44.016648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1442/2846] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:43 2022] PUT /identity/v3/projects/2fa6816d738140b2a9451a0176722cf4/users/7d8b90b0da32476fb0a36b69348bed93/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:21:44.026889 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.027148 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.027455 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.054779 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] MFA Rules not processed for user `7d8b90b0da32476fb0a36b69348bed93`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:44.055997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:44.124693 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1405/2847] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:44.137975 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:44.139814 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:44.201429 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237345d7-b457-4e74-9350-10d9391b784f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:44.206516 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:44.207169 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.207446 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.207732 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.218290 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:44.219753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9bc8b-2f7b-42ba-9230-2fe226239051 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:44.229648 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1443/2848] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:21:44.392098 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:44.462635 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddf2ae-c075-48d8-8195-0d0d102b20d2 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:44.468661 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:44.469606 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.469886 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.470158 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] PATH_INFO: `/v3/users/882258768cfb43da96fe1d896d415894` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.471478 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=882258768cfb43da96fe1d896d415894)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:44.473540 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:44.495690 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:44.522379 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882258768cfb43da96fe1d896d415894'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:44.529794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882258768cfb43da96fe1d896d415894'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:44.539398 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882258768cfb43da96fe1d896d415894'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:44.540023 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882258768cfb43da96fe1d896d415894'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:44.540683 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882258768cfb43da96fe1d896d415894'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:44.558318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1406/2849] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:21:44 2022] DELETE /identity/v3/users/882258768cfb43da96fe1d896d415894 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:44.563085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.563560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.563695 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.583028 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:44.584241 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:44.652777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1444/2850] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:44.662796 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:44.664679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:44.735276 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab540aa-72bc-461a-83be-0b6817181c2a admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:44.740242 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:44.740890 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:44.741210 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:44.741515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:44.751224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:44.752681 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:44.762295 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1407/2851] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:21:45.050171 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:45.051847 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:45.064692 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb7090-68b1-4678-8360-4590df12675c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:45.070100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:45.071181 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.071332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.071557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] PATH_INFO: `/v3/projects/a6dc77d70190462e93932af4f76c2dac` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.073341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dc77d70190462e93932af4f76c2dac)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:45.075592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:45.124390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dc77d70190462e93932af4f76c2dac', 'request_id': 'req-5b97b178-e6d0-48f4-9673-042846222d32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.130505 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dc77d70190462e93932af4f76c2dac', 'request_id': 'req-5b97b178-e6d0-48f4-9673-042846222d32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.136208 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dc77d70190462e93932af4f76c2dac', 'request_id': 'req-5b97b178-e6d0-48f4-9673-042846222d32'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.150741 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] The token cache is being invalidate because project a6dc77d70190462e93932af4f76c2dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:21:45.151061 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.151736 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.153557 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1445/2852] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:21:45 2022] DELETE /identity/v3/projects/a6dc77d70190462e93932af4f76c2dac => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:45.157903 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:45.167633 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e6819-57ee-4dc9-ae76-42f918714089 service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:45.229487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:45.230403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.230679 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.230954 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] PATH_INFO: `/v3/users/7d8b90b0da32476fb0a36b69348bed93` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.232222 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8b90b0da32476fb0a36b69348bed93)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:45.234234 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:45.280011 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:45.282607 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8b90b0da32476fb0a36b69348bed93'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.291150 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8b90b0da32476fb0a36b69348bed93'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.299299 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8b90b0da32476fb0a36b69348bed93'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.299941 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8b90b0da32476fb0a36b69348bed93'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.300584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-ab17e793-fe5c-48f7-b379-f63c6d0a0603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8b90b0da32476fb0a36b69348bed93'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.319273 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1408/2853] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:21:45 2022] DELETE /identity/v3/users/7d8b90b0da32476fb0a36b69348bed93 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:45.597938 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:45.599588 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:45.613577 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97b178-e6d0-48f4-9673-042846222d32 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:45.626768 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:45.683958 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:45.684973 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.685158 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.685466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] PATH_INFO: `/v3/projects/2fa6816d738140b2a9451a0176722cf4` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.686709 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa6816d738140b2a9451a0176722cf4)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:45.688641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:45.737085 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa6816d738140b2a9451a0176722cf4', 'request_id': 'req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.743161 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa6816d738140b2a9451a0176722cf4', 'request_id': 'req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.749025 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa6816d738140b2a9451a0176722cf4', 'request_id': 'req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.763877 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] The token cache is being invalidate because project 2fa6816d738140b2a9451a0176722cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:21:45.764147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.764776 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-c0ff9cdd-06be-4ed5-9101-7d6adf694c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:21:45.766388 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1446/2854] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:21:45 2022] DELETE /identity/v3/projects/2fa6816d738140b2a9451a0176722cf4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:45.772341 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.772477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.772794 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.795988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:45.797223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:45.862741 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1409/2855] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:45.874314 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.874604 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.874879 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.894804 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:45.896023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:45.905273 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1447/2856] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:45.909819 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:45.920154 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56545b6-0375-42f6-9053-fbda52750498 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:45.925943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:45.926632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.926910 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.927178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.927915 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:21:45.928219 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:45.929877 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:45.944984 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1410/2857] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:21:45.950090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:45.960528 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d046312-db2c-4c51-8c23-045dd837e893 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:45.966900 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:45.967560 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:45.967842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:45.968119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:45.968826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:45.970785 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.003275 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1448/2858] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:21:46.007562 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.018783 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a9e13-fa6f-487c-a75c-928ea06ac0a9 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.078444 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.079119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.079381 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.079657 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.080337 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.082292 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.105550 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1411/2859] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:21:46 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:21:46.110522 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.120963 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c95f99-ece2-47cc-b34e-4f4db5b49ebe admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.126351 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.127009 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.127285 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.127550 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.128277 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.129914 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.141710 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1449/2860] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:46.145988 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.157555 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640f20-fe25-4aa1-8e84-b58226f7f148 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.162572 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.163225 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.163515 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.163788 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.164513 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.166149 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c5691-1198-4c41-9a61-2e5bc58604d8 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.176171 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1412/2861] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:46.180492 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.190882 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855ee7f-a64c-47f8-9a10-6ae5bd8690a0 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.195999 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.196782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.197095 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.197859 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] PATH_INFO: `/v3/projects/5b68b18e7f4c40909928a041655b6cd6/users/2e17f1ae0f7a4ed8b5df867ba7d34aa5/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.213066 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b68b18e7f4c40909928a041655b6cd6, user_id=2e17f1ae0f7a4ed8b5df867ba7d34aa5, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.215771 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.233155 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1450/2862] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:46 2022] PUT /identity/v3/projects/5b68b18e7f4c40909928a041655b6cd6/users/2e17f1ae0f7a4ed8b5df867ba7d34aa5/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:46.244789 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.245121 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.245448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.273394 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] MFA Rules not processed for user `2e17f1ae0f7a4ed8b5df867ba7d34aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:46.274608 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:46.287417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:46.347335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1413/2863] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:46.353518 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.400027 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:46.420130 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e56e5-47b2-41e7-a399-d47236cd81d8 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.425894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.426593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.426874 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.427148 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.427852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.429852 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.458317 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1451/2864] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:21:46 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:21:46.462442 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.474522 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac38d6f-d3df-4cb5-b7fe-b510463904ec None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.533349 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.534040 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.534313 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.534584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.535274 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.537246 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.561240 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1414/2865] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:21:46 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:21:46.565707 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.582200 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26867ba4-d018-4092-ab48-728dc173ddf4 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.589711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.590685 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.591485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.591769 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.592499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.594127 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.604756 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1452/2866] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:46.609477 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.619844 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70b36-36bf-4c88-83bc-eae8e50cebf6 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.625297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.626320 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.626653 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.626964 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] PATH_INFO: `/v3/projects/0141de925bf645689894e60e34724029/users/7579128d084f40b2a71b15e18f7d2c65/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.642033 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0141de925bf645689894e60e34724029, user_id=7579128d084f40b2a71b15e18f7d2c65, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.644686 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.660324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1415/2867] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:46 2022] PUT /identity/v3/projects/0141de925bf645689894e60e34724029/users/7579128d084f40b2a71b15e18f7d2c65/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:46.664377 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.674957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2432-885a-4f8d-84f0-77d61830bf21 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.744452 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.745067 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.745353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.745630 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.746349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.747997 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6364eb3-d8a3-4722-90f0-03b01896a3f6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.762883 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1453/2868] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:21:46.768006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.779643 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cbc0c-438a-4321-bd1b-770b1b03d78f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.784632 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.785566 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.785887 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.786151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] PATH_INFO: `/v3/projects/0141de925bf645689894e60e34724029/users/7579128d084f40b2a71b15e18f7d2c65/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.788238 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0141de925bf645689894e60e34724029, user_id=7579128d084f40b2a71b15e18f7d2c65, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:46.790963 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:46.802236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1416/2869] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:21:46 2022] PUT /identity/v3/projects/0141de925bf645689894e60e34724029/users/7579128d084f40b2a71b15e18f7d2c65/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:21:46.812953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.813264 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.813555 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:46.841424 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] MFA Rules not processed for user `7579128d084f40b2a71b15e18f7d2c65`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:21:46.842641 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:46.915570 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1454/2870] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:21:46.927100 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:46.929156 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:46.991324 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86a095-9379-4f10-84d9-38f2d24b6cfe admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:46.995740 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:46.996335 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:46.996614 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:46.996930 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:47.020417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:21:47.060652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:47.062132 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:47.071094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1417/2871] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 28 12:21:52.771261 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:21:52.773046 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:21:52.787687 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00037a-e3c4-4cd6-81db-a83d1323f6c4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:21:52.791974 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:21:52.792614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:21:52.792948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:21:52.793222 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:21:52.803265 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:21:52.804753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cd31d-a3ec-412f-97f5-864c9a2c3a64 service cinder] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:21:52.814330 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1455/2872] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:21:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:21:57.067403 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=17 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:02.810086 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:22.087448 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:22.155402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a782-5297-4cc2-abcf-bc81b129554c service cinder] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:22.160251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:22.161153 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:22.161436 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:22.161710 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] PATH_INFO: `/v3/users/2e17f1ae0f7a4ed8b5df867ba7d34aa5` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:22.163064 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e17f1ae0f7a4ed8b5df867ba7d34aa5)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:22.165110 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:22.216474 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e17f1ae0f7a4ed8b5df867ba7d34aa5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.223284 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e17f1ae0f7a4ed8b5df867ba7d34aa5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.232050 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e17f1ae0f7a4ed8b5df867ba7d34aa5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.232660 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e17f1ae0f7a4ed8b5df867ba7d34aa5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.233321 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e17f1ae0f7a4ed8b5df867ba7d34aa5'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.250520 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1418/2873] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:22:22 2022] DELETE /identity/v3/users/2e17f1ae0f7a4ed8b5df867ba7d34aa5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:22.255178 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:22.255679 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:22.255791 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:22.273759 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:22.274954 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:22.345797 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1456/2874] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:22.355713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:22.357236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:22.420236 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba2557-bc23-4d74-8a7a-0b2075a9bc20 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:22.424541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:22.425224 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:22.425507 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:22.425812 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:22.435876 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:22.437369 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:22.447481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1419/2875] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:22:22.745579 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:22.747187 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:22.760945 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048268ea-e88f-4cba-aeec-1e159facc38c None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:22.766507 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:22.767402 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:22.767686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:22.767956 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] PATH_INFO: `/v3/projects/5b68b18e7f4c40909928a041655b6cd6` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:22.769297 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b68b18e7f4c40909928a041655b6cd6)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:22.771303 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:22.822928 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b68b18e7f4c40909928a041655b6cd6', 'request_id': 'req-2f5755ef-1435-453f-b070-dbe9dfe46744'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.829204 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b68b18e7f4c40909928a041655b6cd6', 'request_id': 'req-2f5755ef-1435-453f-b070-dbe9dfe46744'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.835240 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b68b18e7f4c40909928a041655b6cd6', 'request_id': 'req-2f5755ef-1435-453f-b070-dbe9dfe46744'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.848655 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] The token cache is being invalidate because project 5b68b18e7f4c40909928a041655b6cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:22:22.848878 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.849548 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.851141 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1457/2876] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:22:22 2022] DELETE /identity/v3/projects/5b68b18e7f4c40909928a041655b6cd6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:22.855357 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:22.865297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e0463-0edf-4d52-8f0d-97c4f3e9405a service neutron] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:22.885008 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:22.929198 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:22.930077 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:22.930352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:22.930619 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] PATH_INFO: `/v3/users/7579128d084f40b2a71b15e18f7d2c65` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:22.932151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7579128d084f40b2a71b15e18f7d2c65)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:22.934070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:22.983411 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7579128d084f40b2a71b15e18f7d2c65'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:22.991487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7579128d084f40b2a71b15e18f7d2c65'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.000702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7579128d084f40b2a71b15e18f7d2c65'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.001482 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7579128d084f40b2a71b15e18f7d2c65'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.002114 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d5a5513e-ee96-41d3-accf-34937425827d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7579128d084f40b2a71b15e18f7d2c65'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.019713 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1420/2877] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:22:22 2022] DELETE /identity/v3/users/7579128d084f40b2a71b15e18f7d2c65 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:23.310274 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.311779 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:23.328090 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5755ef-1435-453f-b070-dbe9dfe46744 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.387403 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.388205 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.388485 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.388758 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] PATH_INFO: `/v3/projects/0141de925bf645689894e60e34724029` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.390100 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0141de925bf645689894e60e34724029)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.392021 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.426212 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:23.445633 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0141de925bf645689894e60e34724029', 'request_id': 'req-19d93551-039c-48c5-8e8a-5f8476890929'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.453119 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0141de925bf645689894e60e34724029', 'request_id': 'req-19d93551-039c-48c5-8e8a-5f8476890929'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.459569 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0141de925bf645689894e60e34724029', 'request_id': 'req-19d93551-039c-48c5-8e8a-5f8476890929'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.473499 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] The token cache is being invalidate because project 0141de925bf645689894e60e34724029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80100) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:22:23.473789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.474519 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-19d93551-039c-48c5-8e8a-5f8476890929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:23.476190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1458/2878] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:22:23 2022] DELETE /identity/v3/projects/0141de925bf645689894e60e34724029 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:23.481525 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.482044 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.482166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.501204 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:23.502417 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:23.564704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1421/2879] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:23.573949 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.574236 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.574515 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.595323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:23.596532 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:23.605338 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1459/2880] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 28 12:22:23.610214 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.624574 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562b4a-afa5-4cab-a225-f98a9dbbb454 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.630843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.631509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.631792 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.632067 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] PATH_INFO: `/v3/domains` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.632847 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80099) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:22:23.633179 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.634943 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.647025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1422/2881] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:22:23.651497 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.661423 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de6363-043c-4c2d-a902-3b1588bdbdee None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.667957 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.668614 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.668889 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.669185 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.669871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.671770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.702466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1460/2882] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:22:23 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:22:23.707279 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.720620 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f3e9c-663e-473f-a72f-40cc9982e38f admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.748079 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:23.782086 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.782777 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.783075 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.783356 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.784048 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.786018 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.810692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1423/2883] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:22:23 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:22:23.814595 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.826065 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f2846-fb1e-4c44-86c6-97f201d0763f admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.831432 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.832097 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.832378 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.832639 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.833433 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.835050 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.845811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1461/2884] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:22:23.850014 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.861166 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35aae-e62f-4df6-9d46-27f72f121b89 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.866823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.867480 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.867787 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.868058 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.868755 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.870652 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07248-a748-443a-af34-07474a4a558b admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.880823 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1424/2885] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:22:23.885408 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:23.899777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525afcfb-d349-461d-9cd5-1317d2f158e9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:23.905778 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:23.906671 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.906943 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.907210 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] PATH_INFO: `/v3/projects/a972d7ef7db44253bcda2f5f75e42728/users/0e8dfd7928e7426b8525ae9b8ca9501b/roles/724184bfe3d1497590814ff09b602192` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.923781 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a972d7ef7db44253bcda2f5f75e42728, user_id=0e8dfd7928e7426b8525ae9b8ca9501b, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:23.926513 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:23.943260 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1462/2886] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:22:23 2022] PUT /identity/v3/projects/a972d7ef7db44253bcda2f5f75e42728/users/0e8dfd7928e7426b8525ae9b8ca9501b/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:23.959297 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:23.959585 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:23.959878 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:23.989308 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] MFA Rules not processed for user `0e8dfd7928e7426b8525ae9b8ca9501b`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:23.990532 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:24.061473 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1425/2887] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:24.066754 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.134833 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a70d-016f-42b7-b1be-2b55c89ea9bf admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.140842 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.141541 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.141825 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.142096 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.142789 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.144730 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.173125 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1463/2888] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:22:24 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:22:24.177134 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.187827 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c17c4-e94d-4027-bd2e-bbc37efbaaab None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.246318 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.246985 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.247268 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.247551 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] PATH_INFO: `/v3/users` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.248252 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.250229 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.272025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1426/2889] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:22:24 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 28 12:22:24.276237 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.286796 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa135-c614-40da-9e3d-11aceb5bb940 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.292729 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.293428 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.293711 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.293984 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.294717 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.296327 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.307625 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1464/2890] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:22:24.312070 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.322688 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001c284-01f6-4504-9616-646622f0356c admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.328433 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.329347 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.329623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.329875 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] PATH_INFO: `/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d/users/055cb5c851fd45b4893922f54dc524af/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.346821 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4738f7252e49c5aaa2b3dbfbee8f0d, user_id=055cb5c851fd45b4893922f54dc524af, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.349541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.364772 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1427/2891] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:22:24 2022] PUT /identity/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d/users/055cb5c851fd45b4893922f54dc524af/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 28 12:22:24.368816 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.378940 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bbed4-3067-4a48-81d0-906dc20f77c9 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.398626 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:24.440215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.440809 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.441122 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.441397 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.442128 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.443748 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657f084-e755-4ca5-9bb1-bb10d48fb7cd admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.454103 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1465/2892] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:22:24.458402 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.464443 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:24.470582 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e044f37-8099-42b4-ab41-9b1aca687e51 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.476301 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.477230 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.477514 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.477813 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] PATH_INFO: `/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d/users/055cb5c851fd45b4893922f54dc524af/roles/a034d6cd495d4d39acddae91f37019a3` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.480212 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4738f7252e49c5aaa2b3dbfbee8f0d, user_id=055cb5c851fd45b4893922f54dc524af, role_id=a034d6cd495d4d39acddae91f37019a3)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.482967 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.493999 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1428/2893] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:22:24 2022] PUT /identity/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d/users/055cb5c851fd45b4893922f54dc524af/roles/a034d6cd495d4d39acddae91f37019a3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:24.504414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.504944 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.505258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.534042 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] MFA Rules not processed for user `055cb5c851fd45b4893922f54dc524af`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:24.535263 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:24.603811 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1466/2894] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:24.616744 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.619039 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:24.679857 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e700c-67a9-4e8e-9a32-a9bbb31966f3 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.683702 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.684350 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.684623 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.684896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.694408 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.695799 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0eb14f-72b9-43ae-ae0c-f5c0461b5cce service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.705735 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1429/2895] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:22:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:22:24.735145 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:24.805219 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e71f3e-8217-4773-a538-b631d597ecb4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:24.810574 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:24.811448 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.811725 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.811992 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] PATH_INFO: `/v3/users/0e8dfd7928e7426b8525ae9b8ca9501b` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.813221 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8dfd7928e7426b8525ae9b8ca9501b)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:24.815217 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:24.867370 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8dfd7928e7426b8525ae9b8ca9501b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:24.869058 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:24.878891 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8dfd7928e7426b8525ae9b8ca9501b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:24.886592 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8dfd7928e7426b8525ae9b8ca9501b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:24.887229 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8dfd7928e7426b8525ae9b8ca9501b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:24.887839 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8dfd7928e7426b8525ae9b8ca9501b'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:24.905223 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1467/2896] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:22:24 2022] DELETE /identity/v3/users/0e8dfd7928e7426b8525ae9b8ca9501b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:24.909884 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:24.910178 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:24.910459 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:24.928251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:24.929495 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:24.990747 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1430/2897] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:25.002114 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:25.004480 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:25.066861 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f083-19f8-4e94-95e2-658489a8c2ed admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:25.070843 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:25.071865 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:25.072147 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:25.072414 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:25.081829 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:25.083268 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:25.093156 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1468/2898] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:22:25.375556 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:25.377317 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:25.394481 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775284d-8d75-4b8f-8ebf-293c977c5e6a None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:25.400330 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:25.401781 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:25.402151 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:25.402969 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] PATH_INFO: `/v3/projects/a972d7ef7db44253bcda2f5f75e42728` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:25.405116 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a972d7ef7db44253bcda2f5f75e42728)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:25.407932 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:25.468499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a972d7ef7db44253bcda2f5f75e42728', 'request_id': 'req-d0b065b2-9e21-4444-b00e-ac5846555fdb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.475094 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a972d7ef7db44253bcda2f5f75e42728', 'request_id': 'req-d0b065b2-9e21-4444-b00e-ac5846555fdb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.481261 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a972d7ef7db44253bcda2f5f75e42728', 'request_id': 'req-d0b065b2-9e21-4444-b00e-ac5846555fdb'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.496083 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] The token cache is being invalidate because project a972d7ef7db44253bcda2f5f75e42728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:22:25.496689 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.496997 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.498692 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1431/2899] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:22:25 2022] DELETE /identity/v3/projects/a972d7ef7db44253bcda2f5f75e42728 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:25.504304 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:25.514553 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6563776-9011-48a1-b150-af6f3f8f7c70 service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:25.572578 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:25.573593 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:25.573873 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:25.574146 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] PATH_INFO: `/v3/users/055cb5c851fd45b4893922f54dc524af` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:25.575353 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055cb5c851fd45b4893922f54dc524af)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:25.577390 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:25.625512 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055cb5c851fd45b4893922f54dc524af'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.632969 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055cb5c851fd45b4893922f54dc524af'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.640215 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055cb5c851fd45b4893922f54dc524af'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.640840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055cb5c851fd45b4893922f54dc524af'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.641468 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-7be40cfc-fb00-4c0b-bb19-d7ff6d386ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055cb5c851fd45b4893922f54dc524af'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:25.646542 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:25.659770 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1469/2900] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:22:25 2022] DELETE /identity/v3/users/055cb5c851fd45b4893922f54dc524af => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:25.938673 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:25.940376 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:25.954472 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b065b2-9e21-4444-b00e-ac5846555fdb admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:25.966972 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:26.024038 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.024896 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.025220 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.025485 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] PATH_INFO: `/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.027057 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4738f7252e49c5aaa2b3dbfbee8f0d)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.029042 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.088080 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4738f7252e49c5aaa2b3dbfbee8f0d', 'request_id': 'req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:26.094843 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4738f7252e49c5aaa2b3dbfbee8f0d', 'request_id': 'req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:26.101167 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4738f7252e49c5aaa2b3dbfbee8f0d', 'request_id': 'req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:26.115223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] The token cache is being invalidate because project af4738f7252e49c5aaa2b3dbfbee8f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:22:26.115509 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:26.116189 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-41bdd771-b9c2-4096-ab46-fc8d2df32a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:26.117865 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1432/2901] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:22:25 2022] DELETE /identity/v3/projects/af4738f7252e49c5aaa2b3dbfbee8f0d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:26.123627 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.123917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.124197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.142858 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:26.144072 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:26.207282 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1470/2902] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:26.216391 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.216727 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.217133 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.237107 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:26.238315 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:26.247069 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1433/2903] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:26.251618 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.261826 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8693d-7d71-4b39-8ffa-947cdd0f97d4 None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.267245 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.267903 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.268190 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.268466 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.269249 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80100) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 28 12:22:26.269559 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.271197 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.283545 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1471/2904] 10.222.0.55 () {58 vars in 1152 bytes} [Tue Jun 28 12:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 28 12:22:26.288449 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.299458 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddb4e3-dbc9-488b-a302-2ea58eaad712 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.309025 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.313316 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.313598 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.313899 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.315487 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.317469 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.347346 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1434/2905] 10.222.0.55 () {60 vars in 1152 bytes} [Tue Jun 28 12:22:26 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:22:26.352136 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.363420 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc25c-c470-4b07-8106-a7d031a687d7 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.428077 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.428777 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.429106 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.429400 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] PATH_INFO: `/v3/users` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.430191 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.432189 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.454417 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1472/2906] 10.222.0.55 () {60 vars in 1143 bytes} [Tue Jun 28 12:22:26 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 28 12:22:26.460695 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.475006 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c729754-45c2-4c77-8807-7d3860b0f5c2 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.481311 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.481977 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.482253 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.482543 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.483263 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.484866 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.495554 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1435/2907] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 28 12:22:26.499703 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.510349 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748126-dba4-4082-96e4-44a37c59db00 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.515784 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.516486 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.516772 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.517088 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.517840 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.519412 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f4915-2966-4073-99c6-f2a38af8cbcc admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.530026 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1473/2908] 10.222.0.55 () {58 vars in 1121 bytes} [Tue Jun 28 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 28 12:22:26.534467 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.545579 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86288b8-904d-4f86-8606-7d10ea8d8dc4 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.551476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.552387 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.552662 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.552942 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] PATH_INFO: `/v3/projects/fb2b71ef0f494486898301c7c157dd80/users/c093679524204f3a881e260c7f740a1f/roles/724184bfe3d1497590814ff09b602192` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.568584 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2b71ef0f494486898301c7c157dd80, user_id=c093679524204f3a881e260c7f740a1f, role_id=724184bfe3d1497590814ff09b602192)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.571309 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.589545 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1436/2909] 10.222.0.55 () {60 vars in 1482 bytes} [Tue Jun 28 12:22:26 2022] PUT /identity/v3/projects/fb2b71ef0f494486898301c7c157dd80/users/c093679524204f3a881e260c7f740a1f/roles/724184bfe3d1497590814ff09b602192 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:26.600150 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] REQUEST_METHOD: `POST` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.600443 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.600723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.629807 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.auth.core [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] MFA Rules not processed for user `c093679524204f3a881e260c7f740a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80100) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:26.631023 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:26.673172 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:26.697686 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1474/2910] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:26.713704 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:26.715766 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:26.763476 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:26.781405 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c3a4-36dc-457a-812f-bcd50af5b437 admin admin] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:26.785529 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c561ff3ab84f99b81eaccc9d9f2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:26.786137 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] REQUEST_METHOD: `GET` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:26.786419 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:26.786691 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:26.796218 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:26.797706 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ec512-dc98-41f5-96b6-2fabcd499778 service cinder] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:26.807251 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1437/2911] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:22:36.698923 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=8 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:36.802527 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=7 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:22:49.795917 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:49.873782 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434aab-9b77-4028-9c57-460b9dfe2dca None None] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:49.879079 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:49.879977 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:49.880258 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:49.880527 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] PATH_INFO: `/v3/users/c093679524204f3a881e260c7f740a1f` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:49.882019 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c093679524204f3a881e260c7f740a1f)` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:49.883953 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:49.935925 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c093679524204f3a881e260c7f740a1f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:49.943723 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c093679524204f3a881e260c7f740a1f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:49.951332 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c093679524204f3a881e260c7f740a1f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:49.951980 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c093679524204f3a881e260c7f740a1f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:49.952610 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.notifications [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c093679524204f3a881e260c7f740a1f'} {{(pid=80100) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:49.970008 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1475/2912] 10.222.0.55 () {58 vars in 1223 bytes} [Tue Jun 28 12:22:49 2022] DELETE /identity/v3/users/c093679524204f3a881e260c7f740a1f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:22:49.975760 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] REQUEST_METHOD: `POST` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:49.977352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:49.977352 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:50.000386 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.auth.core [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] MFA Rules not processed for user `02f93a0d97354be4b96d139a23cac2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80099) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 28 12:22:50.001709 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:50.073119 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1438/2913] 10.222.0.55 () {58 vars in 957 bytes} [Tue Jun 28 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 28 12:22:50.086716 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:50.088323 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:50.154871 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa7d24-87fa-4eb0-9362-d1da06091956 admin admin] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:50.158733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872c307ac6c24374b76b61b905d1f4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:50.159395 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:50.159676 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:50.159948 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:50.169320 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:50.170753 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:50.180335 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1476/2914] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:22:50.464499 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] Authenticating user token {{(pid=80099) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:22:50.467223 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.fernet_utils [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80099) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:22:50.483036 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123970bd-39f2-4435-84f6-afb9ad0b3f08 None None] Validating token access rules against request {{(pid=80099) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:22:50.488325 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f93a0d97354be4b96d139a23cac2ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c70d24580e841bb8af0388fc0999482', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80099) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:22:50.489260 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:22:50.489539 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:22:50.489807 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] PATH_INFO: `/v3/projects/fb2b71ef0f494486898301c7c157dd80` {{(pid=80099) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:22:50.491123 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2b71ef0f494486898301c7c157dd80)` {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:22:50.493302 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] RBAC: Authorization granted {{(pid=80099) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:22:50.541629 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2b71ef0f494486898301c7c157dd80', 'request_id': 'req-c452431d-788d-4958-b483-08a7fbd80eb4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:50.547748 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2b71ef0f494486898301c7c157dd80', 'request_id': 'req-c452431d-788d-4958-b483-08a7fbd80eb4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:50.553489 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2b71ef0f494486898301c7c157dd80', 'request_id': 'req-c452431d-788d-4958-b483-08a7fbd80eb4'} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:50.566982 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] The token cache is being invalidate because project fb2b71ef0f494486898301c7c157dd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80099) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 28 12:22:50.567265 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:50.567871 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG keystone.notifications [None req-c452431d-788d-4958-b483-08a7fbd80eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80099) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 28 12:22:50.569541 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: [pid: 80099|app: 0|req: 1439/2915] 10.222.0.55 () {58 vars in 1232 bytes} [Tue Jun 28 12:22:50 2022] DELETE /identity/v3/projects/fb2b71ef0f494486898301c7c157dd80 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 28 12:23:00.176068 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:23:00.564293 dv-cinder-847954-1-iscsi devstack@keystone.service[80099]: DEBUG dbcounter [-] [80099] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80099) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 28 12:23:42.297656 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] Authenticating user token {{(pid=80100) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 28 12:23:42.299286 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:23:42.393894 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08c16-2aeb-42a3-9133-3904038c446c service neutron] Validating token access rules against request {{(pid=80100) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 28 12:23:42.460341 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bb4ac455d464617840790998ceea9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5709e926be584e859153090dd261a990', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80100) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 28 12:23:42.461031 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] REQUEST_METHOD: `GET` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 28 12:23:42.461311 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] SCRIPT_NAME: `/identity` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 28 12:23:42.461581 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80100) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 28 12:23:42.463418 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.fernet_utils [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80100) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 28 12:23:42.476176 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 28 12:23:42.477806 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c14e8-5b48-48fa-b4e4-8c3449c06ab8 service placement] RBAC: Authorization granted {{(pid=80100) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 28 12:23:42.488761 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: [pid: 80100|app: 0|req: 1477/2916] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Jun 28 12:23:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 28 12:23:52.483733 dv-cinder-847954-1-iscsi devstack@keystone.service[80100]: DEBUG dbcounter [-] [80100] Writing DB stats keystone:SELECT=58 {{(pid=80100) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}